Sep 09 22:58:58.544290 ip-10-0-131-92 systemd[1]: Starting Kubernetes Kubelet... Sep 09 22:58:58.960847 ip-10-0-131-92 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 09 22:58:58.960847 ip-10-0-131-92 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 09 22:58:58.960847 ip-10-0-131-92 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 09 22:58:58.960847 ip-10-0-131-92 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 09 22:58:58.960847 ip-10-0-131-92 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 09 22:58:58.961865 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.961755 2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 09 22:58:58.965101 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965085 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 22:58:58.965101 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965100 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 22:58:58.965101 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965104 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 22:58:58.965189 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965107 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 22:58:58.965189 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965110 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 22:58:58.965189 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965113 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 22:58:58.965189 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965116 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 22:58:58.965189 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965119 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 22:58:58.965189 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965121 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 22:58:58.965189 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965124 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 22:58:58.965189 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965129 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 22:58:58.965189 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965132 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 22:58:58.965189 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965135 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 22:58:58.965189 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965138 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 22:58:58.965189 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965141 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 22:58:58.965189 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965144 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 22:58:58.965189 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965147 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 22:58:58.965189 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965149 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 22:58:58.965189 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965152 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 22:58:58.965189 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965154 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 22:58:58.965189 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965172 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 22:58:58.965189 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965176 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 22:58:58.965644 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965179 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 22:58:58.965644 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965182 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 22:58:58.965644 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965185 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 22:58:58.965644 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965188 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 22:58:58.965644 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965191 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 22:58:58.965644 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965195 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 22:58:58.965644 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965198 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 22:58:58.965644 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965201 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 22:58:58.965644 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965204 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 22:58:58.965644 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965207 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 22:58:58.965644 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965210 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 22:58:58.965644 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965212 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 22:58:58.965644 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965214 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 22:58:58.965644 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965217 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 22:58:58.965644 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965220 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 22:58:58.965644 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965222 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 22:58:58.965644 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965226 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 22:58:58.965644 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965229 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 22:58:58.965644 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965231 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 22:58:58.965644 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965234 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 22:58:58.966145 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965236 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 22:58:58.966145 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965239 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 22:58:58.966145 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965241 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 22:58:58.966145 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965244 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 22:58:58.966145 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965247 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 22:58:58.966145 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965249 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 22:58:58.966145 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965252 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 22:58:58.966145 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965254 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 22:58:58.966145 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965256 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 22:58:58.966145 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965259 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 22:58:58.966145 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965262 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 22:58:58.966145 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965265 2580 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 22:58:58.966145 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965268 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 22:58:58.966145 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965271 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 22:58:58.966145 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965273 2580 feature_gate.go:328] unrecognized feature gate: Example Sep 09 22:58:58.966145 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965276 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 22:58:58.966145 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965278 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 22:58:58.966145 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965281 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 22:58:58.966145 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965284 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 22:58:58.966145 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965286 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 22:58:58.966635 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965289 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 22:58:58.966635 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965291 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 22:58:58.966635 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965294 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 22:58:58.966635 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965296 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 22:58:58.966635 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965299 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 22:58:58.966635 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965301 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 22:58:58.966635 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965304 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 22:58:58.966635 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965306 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 22:58:58.966635 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965309 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 22:58:58.966635 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965312 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 22:58:58.966635 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965314 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 22:58:58.966635 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965317 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 22:58:58.966635 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965319 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 22:58:58.966635 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965322 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 22:58:58.966635 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965324 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 22:58:58.966635 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965327 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 22:58:58.966635 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965329 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 22:58:58.966635 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965332 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 22:58:58.966635 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965334 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 22:58:58.967113 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965337 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 22:58:58.967113 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965339 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 22:58:58.967113 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965342 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 22:58:58.967113 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965346 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 22:58:58.967113 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965348 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 22:58:58.967113 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965351 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 22:58:58.967113 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965354 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 22:58:58.967113 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965357 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 22:58:58.967113 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965359 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 22:58:58.967113 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965362 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 22:58:58.967113 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965364 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 22:58:58.967113 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965367 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 22:58:58.967113 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965370 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 22:58:58.967113 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965374 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 22:58:58.967113 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965377 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 22:58:58.967113 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965379 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 22:58:58.967113 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965382 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 22:58:58.967113 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.965385 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 22:58:58.967113 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966073 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 22:58:58.967607 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966080 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 22:58:58.967607 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966084 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 22:58:58.967607 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966087 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 22:58:58.967607 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966090 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 22:58:58.967607 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966093 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 22:58:58.967607 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966096 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 22:58:58.967607 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966099 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 22:58:58.967607 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966102 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 22:58:58.967607 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966104 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 22:58:58.967607 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966107 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 22:58:58.967607 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966110 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 22:58:58.967607 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966113 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 22:58:58.967607 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966115 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 22:58:58.967607 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966118 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 22:58:58.967607 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966120 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 22:58:58.967607 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966124 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 22:58:58.967607 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966128 2580 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 22:58:58.967607 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966131 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 22:58:58.967607 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966134 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 22:58:58.968092 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966137 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 22:58:58.968092 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966140 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 22:58:58.968092 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966144 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 22:58:58.968092 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966146 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 22:58:58.968092 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966150 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 22:58:58.968092 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966153 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 22:58:58.968092 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966156 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 22:58:58.968092 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966159 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 22:58:58.968092 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966162 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 22:58:58.968092 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966164 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 22:58:58.968092 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966167 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 22:58:58.968092 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966169 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 22:58:58.968092 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966172 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 22:58:58.968092 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966174 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 22:58:58.968092 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966177 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 22:58:58.968092 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966179 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 22:58:58.968092 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966182 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 22:58:58.968092 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966185 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 22:58:58.968092 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966187 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 22:58:58.968092 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966190 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 22:58:58.968584 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966193 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 22:58:58.968584 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966195 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 22:58:58.968584 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966198 2580 feature_gate.go:328] unrecognized feature gate: Example Sep 09 22:58:58.968584 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966200 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 22:58:58.968584 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966203 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 22:58:58.968584 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966205 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 22:58:58.968584 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966208 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 22:58:58.968584 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966210 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 22:58:58.968584 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966213 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 22:58:58.968584 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966216 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 22:58:58.968584 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966219 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 22:58:58.968584 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966223 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 22:58:58.968584 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966225 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 22:58:58.968584 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966228 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 22:58:58.968584 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966231 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 22:58:58.968584 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966234 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 22:58:58.968584 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966237 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 22:58:58.968584 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966239 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 22:58:58.968584 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966242 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 22:58:58.968584 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966244 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 22:58:58.969167 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966247 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 22:58:58.969167 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966249 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 22:58:58.969167 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966251 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 22:58:58.969167 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966254 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 22:58:58.969167 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966256 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 22:58:58.969167 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966259 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 22:58:58.969167 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966261 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 22:58:58.969167 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966264 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 22:58:58.969167 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966266 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 22:58:58.969167 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966275 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 22:58:58.969167 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966278 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 22:58:58.969167 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966282 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 22:58:58.969167 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966285 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 22:58:58.969167 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966287 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 22:58:58.969167 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966290 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 22:58:58.969167 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966293 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 22:58:58.969167 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966296 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 22:58:58.969167 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966299 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 22:58:58.969167 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966303 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 22:58:58.969167 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966307 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 22:58:58.969718 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966310 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 22:58:58.969718 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966313 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 22:58:58.969718 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966315 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 22:58:58.969718 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966318 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 22:58:58.969718 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966321 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 22:58:58.969718 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966323 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 22:58:58.969718 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966326 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 22:58:58.969718 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966329 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 22:58:58.969718 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966331 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 22:58:58.969718 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966334 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 22:58:58.969718 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966337 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 22:58:58.969718 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966340 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 22:58:58.969718 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966342 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 22:58:58.969718 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966345 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 22:58:58.969718 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966347 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 22:58:58.969718 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966350 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 22:58:58.969718 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966352 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 22:58:58.969718 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966355 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 22:58:58.969718 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966358 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 22:58:58.970253 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966710 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 22:58:58.970253 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966714 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 22:58:58.970253 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966717 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 22:58:58.970253 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966719 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 22:58:58.970253 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966722 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 22:58:58.970253 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966725 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 22:58:58.970253 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966728 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 22:58:58.970253 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966730 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 22:58:58.970253 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966733 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 22:58:58.970253 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966735 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 22:58:58.970253 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966739 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 22:58:58.970253 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966743 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 22:58:58.970253 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966745 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 22:58:58.970253 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966748 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 22:58:58.970253 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966751 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 22:58:58.970253 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966754 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 22:58:58.970253 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966757 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 22:58:58.970253 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966760 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 22:58:58.970253 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966762 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 22:58:58.970868 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966766 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 22:58:58.970868 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966770 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 22:58:58.970868 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966774 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 22:58:58.970868 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966777 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 22:58:58.970868 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966780 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 22:58:58.970868 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966783 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 22:58:58.970868 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966786 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 22:58:58.970868 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966789 2580 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 22:58:58.970868 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966791 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 22:58:58.970868 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966795 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 22:58:58.970868 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966797 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 22:58:58.970868 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966800 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 22:58:58.970868 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966803 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 22:58:58.970868 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966806 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 22:58:58.970868 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966809 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 22:58:58.970868 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966811 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 22:58:58.970868 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966814 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 22:58:58.970868 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966817 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 22:58:58.970868 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966820 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 22:58:58.971412 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966823 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 22:58:58.971412 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966825 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 22:58:58.971412 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966829 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 22:58:58.971412 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966831 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 22:58:58.971412 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966834 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 22:58:58.971412 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966838 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 22:58:58.971412 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966840 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 22:58:58.971412 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966843 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 22:58:58.971412 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966845 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 22:58:58.971412 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966848 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 22:58:58.971412 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966851 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 22:58:58.971412 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966854 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 22:58:58.971412 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966856 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 22:58:58.971412 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966859 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 22:58:58.971412 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966861 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 22:58:58.971412 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966864 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 22:58:58.971412 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966866 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 22:58:58.971412 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966869 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 22:58:58.971412 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966871 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 22:58:58.971412 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966874 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 22:58:58.971944 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966876 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 22:58:58.971944 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966879 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 22:58:58.971944 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966881 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 22:58:58.971944 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966884 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 22:58:58.971944 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966886 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 22:58:58.971944 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966889 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 22:58:58.971944 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966893 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 22:58:58.971944 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966896 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 22:58:58.971944 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966898 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 22:58:58.971944 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966901 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 22:58:58.971944 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966903 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 22:58:58.971944 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966906 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 22:58:58.971944 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966910 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 22:58:58.971944 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966912 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 22:58:58.971944 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966915 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 22:58:58.971944 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966918 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 22:58:58.971944 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966920 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 22:58:58.971944 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966923 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 22:58:58.971944 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966925 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 22:58:58.971944 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966928 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 22:58:58.972501 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966931 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 22:58:58.972501 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966933 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 22:58:58.972501 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966936 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 22:58:58.972501 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966938 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 22:58:58.972501 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966941 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 22:58:58.972501 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966943 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 22:58:58.972501 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966945 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 22:58:58.972501 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966948 2580 feature_gate.go:328] unrecognized feature gate: Example Sep 09 22:58:58.972501 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966950 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 22:58:58.972501 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966953 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 22:58:58.972501 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966955 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 22:58:58.972501 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966958 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 22:58:58.972501 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966960 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 22:58:58.972501 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966963 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 22:58:58.972501 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966966 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 22:58:58.972501 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966968 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 22:58:58.972501 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966984 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 22:58:58.972501 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966987 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 22:58:58.972501 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966991 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 22:58:58.972501 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966993 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 22:58:58.973094 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.966996 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 22:58:58.973094 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967070 2580 flags.go:64] FLAG: --address="0.0.0.0" Sep 09 22:58:58.973094 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967111 2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 09 22:58:58.973094 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967117 2580 flags.go:64] FLAG: --anonymous-auth="true" Sep 09 22:58:58.973094 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967121 2580 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 09 22:58:58.973094 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967126 2580 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 09 22:58:58.973094 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967130 2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 09 22:58:58.973094 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967134 2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 09 22:58:58.973094 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967138 2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 09 22:58:58.973094 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967141 2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 09 22:58:58.973094 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967144 2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 09 22:58:58.973094 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967148 2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 09 22:58:58.973094 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967152 2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 09 22:58:58.973094 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967155 2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 09 22:58:58.973094 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967158 2580 flags.go:64] FLAG: --cgroup-root="" Sep 09 22:58:58.973094 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967160 2580 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 09 22:58:58.973094 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967163 2580 flags.go:64] FLAG: --client-ca-file="" Sep 09 22:58:58.973094 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967166 2580 flags.go:64] FLAG: --cloud-provider="external" Sep 09 22:58:58.973094 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967169 2580 flags.go:64] FLAG: --cluster-dns="[]" Sep 09 22:58:58.973094 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967174 2580 flags.go:64] FLAG: --cluster-domain="" Sep 09 22:58:58.973094 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967176 2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 09 22:58:58.973094 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967179 2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 09 22:58:58.973094 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967182 2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 09 22:58:58.973094 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967185 2580 flags.go:64] FLAG: --container-log-max-files="5" Sep 09 22:58:58.973691 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967189 2580 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 09 22:58:58.973691 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967192 2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 09 22:58:58.973691 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967195 2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 09 22:58:58.973691 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967199 2580 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 09 22:58:58.973691 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967202 2580 flags.go:64] FLAG: --contention-profiling="false" Sep 09 22:58:58.973691 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967204 2580 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 09 22:58:58.973691 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967209 2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 09 22:58:58.973691 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967212 2580 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 09 22:58:58.973691 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967215 2580 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 09 22:58:58.973691 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967219 2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 09 22:58:58.973691 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967222 2580 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 09 22:58:58.973691 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967225 2580 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 09 22:58:58.973691 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967227 2580 flags.go:64] FLAG: --enable-load-reader="false" Sep 09 22:58:58.973691 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967230 2580 flags.go:64] FLAG: --enable-server="true" Sep 09 22:58:58.973691 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967234 2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 09 22:58:58.973691 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967238 2580 flags.go:64] FLAG: --event-burst="100" Sep 09 22:58:58.973691 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967241 2580 flags.go:64] FLAG: --event-qps="50" Sep 09 22:58:58.973691 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967244 2580 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 09 22:58:58.973691 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967247 2580 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 09 22:58:58.973691 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967250 2580 flags.go:64] FLAG: --eviction-hard="" Sep 09 22:58:58.973691 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967254 2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 09 22:58:58.973691 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967257 2580 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 09 22:58:58.973691 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967259 2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 09 22:58:58.973691 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967262 2580 flags.go:64] FLAG: --eviction-soft="" Sep 09 22:58:58.973691 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967265 2580 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 09 22:58:58.974324 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967268 2580 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 09 22:58:58.974324 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967271 2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 09 22:58:58.974324 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967274 2580 flags.go:64] FLAG: --experimental-mounter-path="" Sep 09 22:58:58.974324 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967277 2580 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 09 22:58:58.974324 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967279 2580 flags.go:64] FLAG: --fail-swap-on="true" Sep 09 22:58:58.974324 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967282 2580 flags.go:64] FLAG: --feature-gates="" Sep 09 22:58:58.974324 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967286 2580 flags.go:64] FLAG: --file-check-frequency="20s" Sep 09 22:58:58.974324 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967289 2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 09 22:58:58.974324 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967292 2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 09 22:58:58.974324 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967295 2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 09 22:58:58.974324 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967298 2580 flags.go:64] FLAG: --healthz-port="10248" Sep 09 22:58:58.974324 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967301 2580 flags.go:64] FLAG: --help="false" Sep 09 22:58:58.974324 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967304 2580 flags.go:64] FLAG: --hostname-override="ip-10-0-131-92.ec2.internal" Sep 09 22:58:58.974324 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967309 2580 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 09 22:58:58.974324 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967312 2580 flags.go:64] FLAG: --http-check-frequency="20s" Sep 09 22:58:58.974324 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967315 2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 09 22:58:58.974324 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967318 2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 09 22:58:58.974324 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967322 2580 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 09 22:58:58.974324 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967324 2580 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 09 22:58:58.974324 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967327 2580 flags.go:64] FLAG: --image-service-endpoint="" Sep 09 22:58:58.974324 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967330 2580 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 09 22:58:58.974324 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967333 2580 flags.go:64] FLAG: --kube-api-burst="100" Sep 09 22:58:58.974324 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967336 2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 09 22:58:58.974914 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967339 2580 flags.go:64] FLAG: --kube-api-qps="50" Sep 09 22:58:58.974914 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967342 2580 flags.go:64] FLAG: --kube-reserved="" Sep 09 22:58:58.974914 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967345 2580 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 09 22:58:58.974914 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967348 2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 09 22:58:58.974914 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967351 2580 flags.go:64] FLAG: --kubelet-cgroups="" Sep 09 22:58:58.974914 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967354 2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 09 22:58:58.974914 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967357 2580 flags.go:64] FLAG: --lock-file="" Sep 09 22:58:58.974914 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967360 2580 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 09 22:58:58.974914 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967363 2580 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 09 22:58:58.974914 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967366 2580 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 09 22:58:58.974914 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967371 2580 flags.go:64] FLAG: --log-json-split-stream="false" Sep 09 22:58:58.974914 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967374 2580 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 09 22:58:58.974914 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967377 2580 flags.go:64] FLAG: --log-text-split-stream="false" Sep 09 22:58:58.974914 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967380 2580 flags.go:64] FLAG: --logging-format="text" Sep 09 22:58:58.974914 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967383 2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 09 22:58:58.974914 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967386 2580 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 09 22:58:58.974914 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967389 2580 flags.go:64] FLAG: --manifest-url="" Sep 09 22:58:58.974914 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967391 2580 flags.go:64] FLAG: --manifest-url-header="" Sep 09 22:58:58.974914 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967398 2580 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 09 22:58:58.974914 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967401 2580 flags.go:64] FLAG: --max-open-files="1000000" Sep 09 22:58:58.974914 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967405 2580 flags.go:64] FLAG: --max-pods="110" Sep 09 22:58:58.974914 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967408 2580 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 09 22:58:58.974914 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967412 2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 09 22:58:58.974914 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967415 2580 flags.go:64] FLAG: --memory-manager-policy="None" Sep 09 22:58:58.974914 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967418 2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 09 22:58:58.975545 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967421 2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 09 22:58:58.975545 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967425 2580 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 09 22:58:58.975545 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967427 2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 09 22:58:58.975545 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967435 2580 flags.go:64] FLAG: --node-status-max-images="50" Sep 09 22:58:58.975545 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967438 2580 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 09 22:58:58.975545 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967441 2580 flags.go:64] FLAG: --oom-score-adj="-999" Sep 09 22:58:58.975545 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967444 2580 flags.go:64] FLAG: --pod-cidr="" Sep 09 22:58:58.975545 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967448 2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 09 22:58:58.975545 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967453 2580 flags.go:64] FLAG: --pod-manifest-path="" Sep 09 22:58:58.975545 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967456 2580 flags.go:64] FLAG: --pod-max-pids="-1" Sep 09 22:58:58.975545 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967459 2580 flags.go:64] FLAG: --pods-per-core="0" Sep 09 22:58:58.975545 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967462 2580 flags.go:64] FLAG: --port="10250" Sep 09 22:58:58.975545 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967465 2580 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 09 22:58:58.975545 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967468 2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a4b10669aed538cc" Sep 09 22:58:58.975545 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967471 2580 flags.go:64] FLAG: --qos-reserved="" Sep 09 22:58:58.975545 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967474 2580 flags.go:64] FLAG: --read-only-port="10255" Sep 09 22:58:58.975545 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967477 2580 flags.go:64] FLAG: --register-node="true" Sep 09 22:58:58.975545 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967480 2580 flags.go:64] FLAG: --register-with-taints="" Sep 09 22:58:58.975545 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967483 2580 flags.go:64] FLAG: --registry-burst="10" Sep 09 22:58:58.975545 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967487 2580 flags.go:64] FLAG: --registry-qps="5" Sep 09 22:58:58.975545 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967489 2580 flags.go:64] FLAG: --reserved-cpus="" Sep 09 22:58:58.975545 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967492 2580 flags.go:64] FLAG: --reserved-memory="" Sep 09 22:58:58.975545 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967496 2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 09 22:58:58.975545 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967498 2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 09 22:58:58.975545 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967501 2580 flags.go:64] FLAG: --rotate-certificates="false" Sep 09 22:58:58.976187 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967504 2580 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 09 22:58:58.976187 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967507 2580 flags.go:64] FLAG: --runonce="false" Sep 09 22:58:58.976187 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967509 2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 09 22:58:58.976187 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967512 2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 09 22:58:58.976187 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967516 2580 flags.go:64] FLAG: --seccomp-default="false" Sep 09 22:58:58.976187 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967519 2580 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 09 22:58:58.976187 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967522 2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 09 22:58:58.976187 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967525 2580 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 09 22:58:58.976187 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967528 2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 09 22:58:58.976187 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967531 2580 flags.go:64] FLAG: --storage-driver-password="root" Sep 09 22:58:58.976187 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967534 2580 flags.go:64] FLAG: --storage-driver-secure="false" Sep 09 22:58:58.976187 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967536 2580 flags.go:64] FLAG: --storage-driver-table="stats" Sep 09 22:58:58.976187 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967539 2580 flags.go:64] FLAG: --storage-driver-user="root" Sep 09 22:58:58.976187 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967542 2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 09 22:58:58.976187 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967547 2580 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 09 22:58:58.976187 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967551 2580 flags.go:64] FLAG: --system-cgroups="" Sep 09 22:58:58.976187 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967554 2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 09 22:58:58.976187 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967559 2580 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 09 22:58:58.976187 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967562 2580 flags.go:64] FLAG: --tls-cert-file="" Sep 09 22:58:58.976187 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967565 2580 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 09 22:58:58.976187 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967569 2580 flags.go:64] FLAG: --tls-min-version="" Sep 09 22:58:58.976187 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967571 2580 flags.go:64] FLAG: --tls-private-key-file="" Sep 09 22:58:58.976187 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967574 2580 flags.go:64] FLAG: --topology-manager-policy="none" Sep 09 22:58:58.976187 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967577 2580 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 09 22:58:58.976187 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967580 2580 flags.go:64] FLAG: --topology-manager-scope="container" Sep 09 22:58:58.976847 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967583 2580 flags.go:64] FLAG: --v="2" Sep 09 22:58:58.976847 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967587 2580 flags.go:64] FLAG: --version="false" Sep 09 22:58:58.976847 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967591 2580 flags.go:64] FLAG: --vmodule="" Sep 09 22:58:58.976847 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967595 2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 09 22:58:58.976847 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.967598 2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 09 22:58:58.976847 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967711 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 22:58:58.976847 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967715 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 22:58:58.976847 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967718 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 22:58:58.976847 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967721 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 22:58:58.976847 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967724 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 22:58:58.976847 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967728 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 22:58:58.976847 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967732 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 22:58:58.976847 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967735 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 22:58:58.976847 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967738 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 22:58:58.976847 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967741 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 22:58:58.976847 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967743 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 22:58:58.976847 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967746 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 22:58:58.976847 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967749 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 22:58:58.976847 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967751 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 22:58:58.976847 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967754 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 22:58:58.976847 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967756 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 22:58:58.977638 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967760 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 22:58:58.977638 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967762 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 22:58:58.977638 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967766 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 22:58:58.977638 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967769 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 22:58:58.977638 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967771 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 22:58:58.977638 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967774 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 22:58:58.977638 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967776 2580 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 22:58:58.977638 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967779 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 22:58:58.977638 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967781 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 22:58:58.977638 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967784 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 22:58:58.977638 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967786 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 22:58:58.977638 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967789 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 22:58:58.977638 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967791 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 22:58:58.977638 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967794 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 22:58:58.977638 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967796 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 22:58:58.977638 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967799 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 22:58:58.977638 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967801 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 22:58:58.977638 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967804 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 22:58:58.977638 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967806 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 22:58:58.978487 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967809 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 22:58:58.978487 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967811 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 22:58:58.978487 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967814 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 22:58:58.978487 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967817 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 22:58:58.978487 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967819 2580 feature_gate.go:328] unrecognized feature gate: Example Sep 09 22:58:58.978487 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967822 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 22:58:58.978487 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967824 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 22:58:58.978487 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967827 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 22:58:58.978487 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967830 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 22:58:58.978487 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967832 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 22:58:58.978487 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967835 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 22:58:58.978487 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967837 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 22:58:58.978487 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967839 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 22:58:58.978487 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967843 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 22:58:58.978487 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967846 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 22:58:58.978487 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967848 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 22:58:58.978487 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967852 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 22:58:58.978487 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967854 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 22:58:58.978487 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967857 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 22:58:58.978487 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967859 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 22:58:58.979353 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967862 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 22:58:58.979353 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967864 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 22:58:58.979353 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967867 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 22:58:58.979353 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967870 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 22:58:58.979353 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967872 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 22:58:58.979353 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967874 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 22:58:58.979353 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967877 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 22:58:58.979353 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967879 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 22:58:58.979353 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967882 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 22:58:58.979353 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967884 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 22:58:58.979353 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967887 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 22:58:58.979353 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967890 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 22:58:58.979353 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967892 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 22:58:58.979353 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967895 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 22:58:58.979353 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967897 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 22:58:58.979353 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967900 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 22:58:58.979353 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967902 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 22:58:58.979353 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967905 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 22:58:58.979353 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967908 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 22:58:58.979353 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967910 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 22:58:58.980214 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967913 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 22:58:58.980214 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967916 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 22:58:58.980214 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967918 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 22:58:58.980214 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967921 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 22:58:58.980214 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967923 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 22:58:58.980214 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967927 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 22:58:58.980214 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967930 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 22:58:58.980214 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967933 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 22:58:58.980214 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967935 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 22:58:58.980214 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967938 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 22:58:58.980214 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967941 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 22:58:58.980214 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967945 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 22:58:58.980214 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967949 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 22:58:58.980214 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967953 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 22:58:58.980214 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967955 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 22:58:58.980214 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967958 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 22:58:58.980214 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967961 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 22:58:58.980214 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967964 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 22:58:58.980214 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967967 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 22:58:58.981067 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967983 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 22:58:58.981067 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967988 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 22:58:58.981067 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967992 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 22:58:58.981067 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967996 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 22:58:58.981067 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.967999 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 22:58:58.981067 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.968005 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 09 22:58:58.981067 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.980094 2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 09 22:58:58.981067 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.980117 2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 09 22:58:58.981067 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.980137 2580 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 09 22:58:58.981067 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.980146 2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 09 22:58:58.981067 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980242 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 22:58:58.981067 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980252 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 22:58:58.981067 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980258 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 22:58:58.981067 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980264 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 22:58:58.981719 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980269 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 22:58:58.981719 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980274 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 22:58:58.981719 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980278 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 22:58:58.981719 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980282 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 22:58:58.981719 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980286 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 22:58:58.981719 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980290 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 22:58:58.981719 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980294 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 22:58:58.981719 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980299 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 22:58:58.981719 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980303 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 22:58:58.981719 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980307 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 22:58:58.981719 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980311 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 22:58:58.981719 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980315 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 22:58:58.981719 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980319 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 22:58:58.981719 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980323 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 22:58:58.981719 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980328 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 22:58:58.981719 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980332 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 22:58:58.981719 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980337 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 22:58:58.981719 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980341 2580 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 22:58:58.981719 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980345 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 22:58:58.981719 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980349 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 22:58:58.982332 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980353 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 22:58:58.982332 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980357 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 22:58:58.982332 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980362 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 22:58:58.982332 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980366 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 22:58:58.982332 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980371 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 22:58:58.982332 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980384 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 22:58:58.982332 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980389 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 22:58:58.982332 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980393 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 22:58:58.982332 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980398 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 22:58:58.982332 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980402 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 22:58:58.982332 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980406 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 22:58:58.982332 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980410 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 22:58:58.982332 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980414 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 22:58:58.982332 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980418 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 22:58:58.982332 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980422 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 22:58:58.982332 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980426 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 22:58:58.982332 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980430 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 22:58:58.982332 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980434 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 22:58:58.982332 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980439 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 22:58:58.983103 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980443 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 22:58:58.983103 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980447 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 22:58:58.983103 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980451 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 22:58:58.983103 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980455 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 22:58:58.983103 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980459 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 22:58:58.983103 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980463 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 22:58:58.983103 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980467 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 22:58:58.983103 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980471 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 22:58:58.983103 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980475 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 22:58:58.983103 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980479 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 22:58:58.983103 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980484 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 22:58:58.983103 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980490 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 22:58:58.983103 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980497 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 22:58:58.983103 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980501 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 22:58:58.983103 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980506 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 22:58:58.983103 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980509 2580 feature_gate.go:328] unrecognized feature gate: Example Sep 09 22:58:58.983103 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980522 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 22:58:58.983103 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980527 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 22:58:58.983103 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980531 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 22:58:58.983103 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980545 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 22:58:58.983936 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980550 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 22:58:58.983936 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980554 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 22:58:58.983936 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980559 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 22:58:58.983936 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980563 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 22:58:58.983936 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980567 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 22:58:58.983936 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980571 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 22:58:58.983936 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980575 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 22:58:58.983936 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980579 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 22:58:58.983936 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980583 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 22:58:58.983936 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980587 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 22:58:58.983936 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980591 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 22:58:58.983936 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980595 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 22:58:58.983936 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980599 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 22:58:58.983936 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980603 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 22:58:58.983936 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980607 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 22:58:58.983936 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980611 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 22:58:58.983936 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980615 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 22:58:58.983936 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980619 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 22:58:58.983936 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980623 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 22:58:58.983936 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980627 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 22:58:58.984478 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980634 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 22:58:58.984478 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980640 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 22:58:58.984478 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980644 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 22:58:58.984478 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980648 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 22:58:58.984478 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980653 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 22:58:58.984478 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980657 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 22:58:58.984478 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980661 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 22:58:58.984478 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980665 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 22:58:58.984478 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980669 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 22:58:58.984478 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980674 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 22:58:58.984478 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980679 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 22:58:58.984478 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980683 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 22:58:58.984478 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980695 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 22:58:58.984478 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980700 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 22:58:58.984478 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980704 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 22:58:58.984478 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980708 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 22:58:58.985007 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.980717 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 09 22:58:58.985007 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980941 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 22:58:58.985007 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980949 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 22:58:58.985007 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980954 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 22:58:58.985007 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980958 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 22:58:58.985007 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980963 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 22:58:58.985007 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980967 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 22:58:58.985007 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980988 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 22:58:58.985007 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980993 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 22:58:58.985007 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.980997 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 22:58:58.985007 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981002 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 22:58:58.985007 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981006 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 22:58:58.985007 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981011 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 22:58:58.985007 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981015 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 22:58:58.985007 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981020 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 22:58:58.985403 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981023 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 22:58:58.985403 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981027 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 22:58:58.985403 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981032 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 22:58:58.985403 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981037 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 22:58:58.985403 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981041 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 22:58:58.985403 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981047 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 22:58:58.985403 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981052 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 22:58:58.985403 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981056 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 22:58:58.985403 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981060 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 22:58:58.985403 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981064 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 22:58:58.985403 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981068 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 22:58:58.985403 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981075 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 22:58:58.985403 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981081 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 22:58:58.985403 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981086 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 22:58:58.985403 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981100 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 22:58:58.985403 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981106 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 22:58:58.985403 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981110 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 22:58:58.985403 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981115 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 22:58:58.985403 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981119 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 22:58:58.985869 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981123 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 22:58:58.985869 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981128 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 22:58:58.985869 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981132 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 22:58:58.985869 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981136 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 22:58:58.985869 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981140 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 22:58:58.985869 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981144 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 22:58:58.985869 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981148 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 22:58:58.985869 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981152 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 22:58:58.985869 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981156 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 22:58:58.985869 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981161 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 22:58:58.985869 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981165 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 22:58:58.985869 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981169 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 22:58:58.985869 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981173 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 22:58:58.985869 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981177 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 22:58:58.985869 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981181 2580 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 22:58:58.985869 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981185 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 22:58:58.985869 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981189 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 22:58:58.985869 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981193 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 22:58:58.985869 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981197 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 22:58:58.985869 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981201 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 22:58:58.986436 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981205 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 22:58:58.986436 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981210 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 22:58:58.986436 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981214 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 22:58:58.986436 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981218 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 22:58:58.986436 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981223 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 22:58:58.986436 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981228 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 22:58:58.986436 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981232 2580 feature_gate.go:328] unrecognized feature gate: Example Sep 09 22:58:58.986436 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981236 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 22:58:58.986436 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981248 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 22:58:58.986436 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981252 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 22:58:58.986436 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981256 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 22:58:58.986436 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981260 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 22:58:58.986436 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981264 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 22:58:58.986436 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981268 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 22:58:58.986436 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981272 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 22:58:58.986436 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981277 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 22:58:58.986436 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981280 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 22:58:58.986436 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981285 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 22:58:58.986436 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981289 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 22:58:58.986436 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981293 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 22:58:58.986945 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981297 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 22:58:58.986945 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981301 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 22:58:58.986945 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981304 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 22:58:58.986945 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981309 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 22:58:58.986945 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981313 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 22:58:58.986945 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981317 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 22:58:58.986945 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981327 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 22:58:58.986945 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981334 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 22:58:58.986945 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981340 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 22:58:58.986945 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981344 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 22:58:58.986945 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981350 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 22:58:58.986945 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981354 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 22:58:58.986945 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981359 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 22:58:58.986945 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981363 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 22:58:58.986945 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981368 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 22:58:58.986945 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981372 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 22:58:58.986945 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981377 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 22:58:58.986945 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981381 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 22:58:58.986945 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981385 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 22:58:58.987442 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981389 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 22:58:58.987442 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981393 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 22:58:58.987442 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981406 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 22:58:58.987442 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981412 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 22:58:58.987442 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981416 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 22:58:58.987442 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981420 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 22:58:58.987442 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:58:58.981424 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 22:58:58.987442 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.981432 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 09 22:58:58.987442 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.981618 2580 server.go:962] "Client rotation is on, will bootstrap in background" Sep 09 22:58:58.987442 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.984775 2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 09 22:58:58.987442 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.985925 2580 server.go:1019] "Starting client certificate rotation" Sep 09 22:58:58.987442 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.986080 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 09 22:58:58.987442 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.986138 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 09 22:58:58.992692 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.992669 2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 09 22:58:58.996904 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:58.996882 2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 09 22:58:59.013999 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.013966 2580 log.go:25] "Validated CRI v1 runtime API" Sep 09 22:58:59.017501 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.017452 2580 log.go:25] "Validated CRI v1 image API" Sep 09 22:58:59.017698 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.017676 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 09 22:58:59.018472 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.018456 2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 09 22:58:59.023089 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.022921 2580 fs.go:135] Filesystem UUIDs: map[1182bcd3-b638-43dc-95b2-913bcc71578a:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 b5f73111-42d8-433e-9076-e61bd05d453c:/dev/nvme0n1p4] Sep 09 22:58:59.023180 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.023090 2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 09 22:58:59.029554 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.029426 2580 manager.go:216] Machine: {Timestamp:2026-09-09 22:58:59.027996124 +0000 UTC m=+0.377863149 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099930 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec28044c6274a32241d7a19a8d916867 SystemUUID:ec28044c-6274-a322-41d7-a19a8d916867 BootID:30a96d3c-bd0e-4b3a-9817-9412a35271cd Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:77:31:94:18:f3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:77:31:94:18:f3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:62:7f:47:55:88:da Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 09 22:58:59.029554 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.029544 2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 09 22:58:59.029728 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.029659 2580 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 09 22:58:59.033121 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.033092 2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 09 22:58:59.033264 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.033124 2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-92.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 09 22:58:59.034026 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.034014 2580 topology_manager.go:138] "Creating topology manager with none policy" Sep 09 22:58:59.034065 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.034027 2580 container_manager_linux.go:309] "Creating device plugin manager" Sep 09 22:58:59.034065 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.034036 2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 09 22:58:59.034925 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.034913 2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 09 22:58:59.036783 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.036772 2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 09 22:58:59.041472 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.041457 2580 state_mem.go:36] "Initialized new in-memory state store" Sep 09 22:58:59.041585 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.041575 2580 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 09 22:58:59.041633 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.041612 2580 kubelet.go:486] "Attempting to sync node with API server" Sep 09 22:58:59.041662 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.041633 2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 09 22:58:59.041662 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.041648 2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 09 22:58:59.041662 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.041657 2580 kubelet.go:398] "Adding apiserver pod source" Sep 09 22:58:59.041781 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.041668 2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 09 22:58:59.042217 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.042195 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 09 22:58:59.042300 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.042226 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 09 22:58:59.044856 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.044828 2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 09 22:58:59.050006 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.049956 2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 09 22:58:59.050109 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.050030 2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 09 22:58:59.050362 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.050345 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 09 22:58:59.050544 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.050534 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 09 22:58:59.050614 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.050606 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 09 22:58:59.050682 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.050674 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 09 22:58:59.050749 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.050741 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 09 22:58:59.050816 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.050806 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 09 22:58:59.050961 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.050952 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 09 22:58:59.051042 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.051033 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 09 22:58:59.051133 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.051118 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 09 22:58:59.051221 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.051137 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 09 22:58:59.051221 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.051150 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 09 22:58:59.051221 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.051162 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 09 22:58:59.052403 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.052383 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 09 22:58:59.052478 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.052420 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 09 22:58:59.053276 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:58:59.053246 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-92.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 09 22:58:59.053276 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:58:59.053248 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 09 22:58:59.054699 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.054682 2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-92.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 09 22:58:59.056660 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.056645 2580 server.go:1268] "Started kubelet" Sep 09 22:58:59.056751 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.056705 2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 09 22:58:59.056794 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.056727 2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 09 22:58:59.056843 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.056824 2580 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 09 22:58:59.057575 ip-10-0-131-92 systemd[1]: Started Kubernetes Kubelet. Sep 09 22:58:59.058324 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.058307 2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 09 22:58:59.059502 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.059487 2580 server.go:310] "Adding debug handlers to kubelet server" Sep 09 22:58:59.061539 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:58:59.061520 2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 09 22:58:59.063170 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.063147 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ztk9j" Sep 09 22:58:59.064480 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.064466 2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 09 22:58:59.064548 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.064483 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 09 22:58:59.064957 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:58:59.064741 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-92.ec2.internal\" not found" Sep 09 22:58:59.066839 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.065398 2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 09 22:58:59.066839 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.065465 2580 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 09 22:58:59.066839 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.065477 2580 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 09 22:58:59.066839 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.065593 2580 reconstruct.go:97] "Volume reconstruction finished" Sep 09 22:58:59.066839 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.065602 2580 reconciler.go:29] "Reconciler: start to sync state" Sep 09 22:58:59.066839 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:58:59.065411 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-92.ec2.internal.18d3c93c9682f204 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-92.ec2.internal,UID:ip-10-0-131-92.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-92.ec2.internal,},FirstTimestamp:2026-09-09 22:58:59.056603652 +0000 UTC m=+0.406470676,LastTimestamp:2026-09-09 22:58:59.056603652 +0000 UTC m=+0.406470676,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-92.ec2.internal,}" Sep 09 22:58:59.067681 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.067660 2580 factory.go:55] Registering systemd factory Sep 09 22:58:59.067681 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.067681 2580 factory.go:223] Registration of the systemd container factory successfully Sep 09 22:58:59.068066 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.068043 2580 factory.go:153] Registering CRI-O factory Sep 09 22:58:59.068066 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.068058 2580 factory.go:223] Registration of the crio container factory successfully Sep 09 22:58:59.068187 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.068116 2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 09 22:58:59.068187 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.068128 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ztk9j" Sep 09 22:58:59.068187 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.068139 2580 factory.go:103] Registering Raw factory Sep 09 22:58:59.068187 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.068153 2580 manager.go:1196] Started watching for new ooms in manager Sep 09 22:58:59.068634 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.068581 2580 manager.go:349] Starting recovery of all containers Sep 09 22:58:59.075684 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:58:59.071371 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 09 22:58:59.075684 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:58:59.071469 2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-92.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 09 22:58:59.078994 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.078959 2580 manager.go:354] Recovery completed Sep 09 22:58:59.081277 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:58:59.081247 2580 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Sep 09 22:58:59.085697 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.085684 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 22:58:59.087524 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.087503 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-92.ec2.internal" event="NodeHasSufficientMemory" Sep 09 22:58:59.087615 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.087531 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-92.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 22:58:59.087615 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.087546 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-92.ec2.internal" event="NodeHasSufficientPID" Sep 09 22:58:59.088181 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.088166 2580 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 09 22:58:59.088181 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.088179 2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 09 22:58:59.088327 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.088187 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 22:58:59.088327 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.088198 2580 state_mem.go:36] "Initialized new in-memory state store" Sep 09 22:58:59.089016 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.088998 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-92.ec2.internal" event="NodeHasSufficientMemory" Sep 09 22:58:59.089078 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.089026 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-92.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 22:58:59.089078 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.089036 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-92.ec2.internal" event="NodeHasSufficientPID" Sep 09 22:58:59.090371 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.090358 2580 policy_none.go:49] "None policy: Start" Sep 09 22:58:59.090406 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.090375 2580 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 09 22:58:59.090406 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.090385 2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 09 22:58:59.092387 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.092374 2580 policy_none.go:47] "Start" Sep 09 22:58:59.167502 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:58:59.165208 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-92.ec2.internal\" not found" Sep 09 22:58:59.171484 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.171455 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 09 22:58:59.172653 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.172632 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 09 22:58:59.172750 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.172691 2580 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 09 22:58:59.172750 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.172716 2580 kubelet.go:2448] "Starting kubelet main sync loop" Sep 09 22:58:59.172849 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:58:59.172761 2580 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Sep 09 22:58:59.174500 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.174482 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 22:58:59.201614 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.201595 2580 manager.go:337] "Starting Device Plugin manager" Sep 09 22:58:59.201720 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:58:59.201641 2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 09 22:58:59.201720 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.201653 2580 server.go:87] "Starting device plugin registration server" Sep 09 22:58:59.202049 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.202038 2580 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 09 22:58:59.202092 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.202051 2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 09 22:58:59.202177 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.202159 2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 09 22:58:59.202251 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.202243 2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 09 22:58:59.202313 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.202251 2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 09 22:58:59.202750 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:58:59.202728 2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 09 22:58:59.202863 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:58:59.202776 2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-92.ec2.internal\" not found" Sep 09 22:58:59.273606 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.273494 2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-92.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-92.ec2.internal"] Sep 09 22:58:59.273606 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.273574 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 22:58:59.277050 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:58:59.277032 2580 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-131-92.ec2.internal\" not found" node="ip-10-0-131-92.ec2.internal" Sep 09 22:58:59.277050 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.277038 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-92.ec2.internal" event="NodeHasSufficientMemory" Sep 09 22:58:59.277147 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.277071 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-92.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 22:58:59.277147 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.277082 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-92.ec2.internal" event="NodeHasSufficientPID" Sep 09 22:58:59.279285 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.279272 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 22:58:59.279391 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.279377 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-92.ec2.internal" Sep 09 22:58:59.279433 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.279415 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 22:58:59.280138 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.280122 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-92.ec2.internal" event="NodeHasSufficientMemory" Sep 09 22:58:59.280239 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.280146 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-92.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 22:58:59.280239 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.280126 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-92.ec2.internal" event="NodeHasSufficientMemory" Sep 09 22:58:59.280239 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.280157 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-92.ec2.internal" event="NodeHasSufficientPID" Sep 09 22:58:59.280239 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.280178 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-92.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 22:58:59.280239 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.280192 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-92.ec2.internal" event="NodeHasSufficientPID" Sep 09 22:58:59.282133 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.282114 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-92.ec2.internal" Sep 09 22:58:59.282217 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.282147 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 22:58:59.282918 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.282902 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-92.ec2.internal" event="NodeHasSufficientMemory" Sep 09 22:58:59.282985 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.282933 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-92.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 22:58:59.282985 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.282945 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-92.ec2.internal" event="NodeHasSufficientPID" Sep 09 22:58:59.300640 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:58:59.300615 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-92.ec2.internal\" not found" node="ip-10-0-131-92.ec2.internal" Sep 09 22:58:59.302149 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.302132 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 22:58:59.303150 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.303131 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-92.ec2.internal" event="NodeHasSufficientMemory" Sep 09 22:58:59.303237 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.303164 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-92.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 22:58:59.303237 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.303175 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-92.ec2.internal" event="NodeHasSufficientPID" Sep 09 22:58:59.303237 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.303199 2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-92.ec2.internal" Sep 09 22:58:59.304444 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:58:59.304427 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-92.ec2.internal\" not found" node="ip-10-0-131-92.ec2.internal" Sep 09 22:58:59.311131 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.311116 2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-92.ec2.internal" Sep 09 22:58:59.311187 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:58:59.311138 2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-92.ec2.internal\": node \"ip-10-0-131-92.ec2.internal\" not found" Sep 09 22:58:59.335293 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:58:59.335266 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-92.ec2.internal\" not found" Sep 09 22:58:59.367797 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.367768 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8d896ae2507ca9bca5f4fea55fdadcb0-config\") pod \"kube-apiserver-proxy-ip-10-0-131-92.ec2.internal\" (UID: \"8d896ae2507ca9bca5f4fea55fdadcb0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-92.ec2.internal" Sep 09 22:58:59.367869 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.367799 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fa1cb3f75de266c7e0e4d8ba90349b01-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-92.ec2.internal\" (UID: \"fa1cb3f75de266c7e0e4d8ba90349b01\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-92.ec2.internal" Sep 09 22:58:59.367869 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.367817 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fa1cb3f75de266c7e0e4d8ba90349b01-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-92.ec2.internal\" (UID: \"fa1cb3f75de266c7e0e4d8ba90349b01\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-92.ec2.internal" Sep 09 22:58:59.435952 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:58:59.435919 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-92.ec2.internal\" not found" Sep 09 22:58:59.468373 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.468339 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fa1cb3f75de266c7e0e4d8ba90349b01-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-92.ec2.internal\" (UID: \"fa1cb3f75de266c7e0e4d8ba90349b01\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-92.ec2.internal" Sep 09 22:58:59.468471 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.468406 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fa1cb3f75de266c7e0e4d8ba90349b01-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-92.ec2.internal\" (UID: \"fa1cb3f75de266c7e0e4d8ba90349b01\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-92.ec2.internal" Sep 09 22:58:59.468471 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.468438 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fa1cb3f75de266c7e0e4d8ba90349b01-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-92.ec2.internal\" (UID: \"fa1cb3f75de266c7e0e4d8ba90349b01\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-92.ec2.internal" Sep 09 22:58:59.468557 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.468474 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8d896ae2507ca9bca5f4fea55fdadcb0-config\") pod \"kube-apiserver-proxy-ip-10-0-131-92.ec2.internal\" (UID: \"8d896ae2507ca9bca5f4fea55fdadcb0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-92.ec2.internal" Sep 09 22:58:59.468557 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.468482 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fa1cb3f75de266c7e0e4d8ba90349b01-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-92.ec2.internal\" (UID: \"fa1cb3f75de266c7e0e4d8ba90349b01\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-92.ec2.internal" Sep 09 22:58:59.468557 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.468525 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8d896ae2507ca9bca5f4fea55fdadcb0-config\") pod \"kube-apiserver-proxy-ip-10-0-131-92.ec2.internal\" (UID: \"8d896ae2507ca9bca5f4fea55fdadcb0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-92.ec2.internal" Sep 09 22:58:59.536703 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:58:59.536625 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-92.ec2.internal\" not found" Sep 09 22:58:59.603344 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.603302 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-92.ec2.internal" Sep 09 22:58:59.608113 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.607937 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-92.ec2.internal" Sep 09 22:58:59.637230 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:58:59.637201 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-92.ec2.internal\" not found" Sep 09 22:58:59.737679 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:58:59.737639 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-92.ec2.internal\" not found" Sep 09 22:58:59.838161 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:58:59.838078 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-92.ec2.internal\" not found" Sep 09 22:58:59.865410 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.865382 2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 22:58:59.939036 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:58:59.939003 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-92.ec2.internal\" not found" Sep 09 22:58:59.985461 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.985431 2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 09 22:58:59.985838 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.985550 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 09 22:58:59.985838 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:58:59.985581 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 09 22:59:00.039600 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:00.039566 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-92.ec2.internal\" not found" Sep 09 22:59:00.071413 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:00.071381 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-08 22:53:58 +0000 UTC" deadline="2028-06-12 03:34:01.314470736 +0000 UTC" Sep 09 22:59:00.071413 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:00.071410 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15388h35m1.243064416s" Sep 09 22:59:00.139717 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:00.139682 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-92.ec2.internal\" not found" Sep 09 22:59:00.208999 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:00.208962 2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 22:59:00.239801 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:00.239760 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-92.ec2.internal\" not found" Sep 09 22:59:00.310369 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:00.310342 2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 22:59:00.352179 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:59:00.352040 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa1cb3f75de266c7e0e4d8ba90349b01.slice/crio-ef52789e3b7baf25762a03462e08effc3bb27b06aa786b4c6df89788dbd0ce28 WatchSource:0}: Error finding container ef52789e3b7baf25762a03462e08effc3bb27b06aa786b4c6df89788dbd0ce28: Status 404 returned error can't find the container with id ef52789e3b7baf25762a03462e08effc3bb27b06aa786b4c6df89788dbd0ce28 Sep 09 22:59:00.352474 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:59:00.352452 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d896ae2507ca9bca5f4fea55fdadcb0.slice/crio-21fde6e75ff814f098de13b0b77bce64310c1784f96e6bff4d6b372a8c6c5a93 WatchSource:0}: Error finding container 21fde6e75ff814f098de13b0b77bce64310c1784f96e6bff4d6b372a8c6c5a93: Status 404 returned error can't find the container with id 21fde6e75ff814f098de13b0b77bce64310c1784f96e6bff4d6b372a8c6c5a93 Sep 09 22:59:00.357750 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:00.357729 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 22:59:00.365370 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:00.365355 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-92.ec2.internal" Sep 09 22:59:00.376017 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:00.375997 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 09 22:59:00.376714 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:00.376702 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-92.ec2.internal" Sep 09 22:59:00.384409 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:00.384395 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 09 22:59:01.043480 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.043446 2580 apiserver.go:52] "Watching apiserver" Sep 09 22:59:01.049758 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.049728 2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 09 22:59:01.050126 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.050098 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pckv","openshift-cluster-node-tuning-operator/tuned-cb4k2","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-92.ec2.internal","openshift-multus/network-metrics-daemon-8dwkd","openshift-ovn-kubernetes/ovnkube-node-prbnz","openshift-dns/node-resolver-lwqr2","openshift-image-registry/node-ca-jx58q","openshift-multus/multus-additional-cni-plugins-dc58c","openshift-multus/multus-w9lv9","openshift-network-diagnostics/network-check-target-gdhb9","openshift-network-operator/iptables-alerter-7mz5t","kube-system/global-pull-secret-syncer-b2gxt","kube-system/konnectivity-agent-ps2b2","kube-system/kube-apiserver-proxy-ip-10-0-131-92.ec2.internal"] Sep 09 22:59:01.052546 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.052465 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jx58q" Sep 09 22:59:01.054638 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.054470 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ps2b2" Sep 09 22:59:01.055173 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.055152 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 09 22:59:01.055303 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.055284 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 09 22:59:01.055410 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.055304 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 09 22:59:01.055410 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.055351 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-7gnqk\"" Sep 09 22:59:01.058206 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.056510 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 09 22:59:01.058206 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.056959 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-lp6zf\"" Sep 09 22:59:01.058206 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.057215 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 09 22:59:01.058620 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.058251 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dc58c" Sep 09 22:59:01.060316 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.060295 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dwkd" Sep 09 22:59:01.060405 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:01.060361 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dwkd" podUID="7cbe663c-0531-4339-b48b-c4d4c3dd5c2b" Sep 09 22:59:01.060783 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.060757 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 09 22:59:01.060843 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.060826 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 09 22:59:01.060965 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.060945 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 09 22:59:01.061047 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.060765 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 09 22:59:01.061093 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.061045 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 09 22:59:01.061232 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.061216 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-vpwzc\"" Sep 09 22:59:01.063085 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.063060 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.064957 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.064939 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 09 22:59:01.065408 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.065392 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lwqr2" Sep 09 22:59:01.065574 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.065559 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 09 22:59:01.065915 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.065900 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 09 22:59:01.066154 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.065952 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-zrrrc\"" Sep 09 22:59:01.066504 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.066488 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 09 22:59:01.066786 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.066769 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 09 22:59:01.066786 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.066783 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 09 22:59:01.067039 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.067025 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 09 22:59:01.067681 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.067663 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 09 22:59:01.067934 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.067918 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.068305 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.068289 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 09 22:59:01.068562 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.068543 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-dt8l4\"" Sep 09 22:59:01.070163 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.070149 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pckv" Sep 09 22:59:01.072414 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.072397 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 09 22:59:01.073069 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.072775 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-l7h7r\"" Sep 09 22:59:01.073734 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.073294 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 09 22:59:01.073734 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.073450 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 09 22:59:01.073734 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.073547 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-nb5rh\"" Sep 09 22:59:01.073734 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.073683 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 09 22:59:01.075214 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.074745 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.076343 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.076323 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bece1799-8601-448b-9fb5-67fe028e3d7e-ovn-node-metrics-cert\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.076435 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.076356 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dcc18d79-d518-4d89-ac61-0e3e1ff55794-multus-conf-dir\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.076435 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.076380 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/09c46a31-75e9-40a5-856f-4f53ae4b0a3b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dc58c\" (UID: \"09c46a31-75e9-40a5-856f-4f53ae4b0a3b\") " pod="openshift-multus/multus-additional-cni-plugins-dc58c" Sep 09 22:59:01.076435 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.076401 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xttlb\" (UniqueName: \"kubernetes.io/projected/7cbe663c-0531-4339-b48b-c4d4c3dd5c2b-kube-api-access-xttlb\") pod \"network-metrics-daemon-8dwkd\" (UID: \"7cbe663c-0531-4339-b48b-c4d4c3dd5c2b\") " pod="openshift-multus/network-metrics-daemon-8dwkd" Sep 09 22:59:01.076435 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.076423 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-etc-openvswitch\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.076640 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.076442 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dcc18d79-d518-4d89-ac61-0e3e1ff55794-host-var-lib-kubelet\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.076640 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.076463 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dcc18d79-d518-4d89-ac61-0e3e1ff55794-host-run-multus-certs\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.076640 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.076483 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghdz7\" (UniqueName: \"kubernetes.io/projected/09c46a31-75e9-40a5-856f-4f53ae4b0a3b-kube-api-access-ghdz7\") pod \"multus-additional-cni-plugins-dc58c\" (UID: \"09c46a31-75e9-40a5-856f-4f53ae4b0a3b\") " pod="openshift-multus/multus-additional-cni-plugins-dc58c" Sep 09 22:59:01.076640 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.076506 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-log-socket\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.076640 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.076526 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-host-run-ovn-kubernetes\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.076640 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.076546 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-host-cni-bin\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.076640 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.076566 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bece1799-8601-448b-9fb5-67fe028e3d7e-ovnkube-config\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.076640 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.076587 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bece1799-8601-448b-9fb5-67fe028e3d7e-env-overrides\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.076640 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.076607 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bece1799-8601-448b-9fb5-67fe028e3d7e-ovnkube-script-lib\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.076640 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.076630 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dcc18d79-d518-4d89-ac61-0e3e1ff55794-multus-socket-dir-parent\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.077147 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.076653 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-host-kubelet\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.077147 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.076675 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-run-systemd\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.077147 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.076698 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.077147 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.076721 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvjcm\" (UniqueName: \"kubernetes.io/projected/bece1799-8601-448b-9fb5-67fe028e3d7e-kube-api-access-xvjcm\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.077147 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.076741 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dcc18d79-d518-4d89-ac61-0e3e1ff55794-cni-binary-copy\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.077147 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.076771 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2a2906d3-5dc1-441e-b7f8-a97e2a67d4ed-agent-certs\") pod \"konnectivity-agent-ps2b2\" (UID: \"2a2906d3-5dc1-441e-b7f8-a97e2a67d4ed\") " pod="kube-system/konnectivity-agent-ps2b2" Sep 09 22:59:01.077147 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.076792 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wg4w9\" (UniqueName: \"kubernetes.io/projected/dadd077d-e65f-49c0-a1bf-b8299ef2bb89-kube-api-access-wg4w9\") pod \"node-resolver-lwqr2\" (UID: \"dadd077d-e65f-49c0-a1bf-b8299ef2bb89\") " pod="openshift-dns/node-resolver-lwqr2" Sep 09 22:59:01.077147 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.076813 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dcc18d79-d518-4d89-ac61-0e3e1ff55794-cnibin\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.077147 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.076834 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-var-lib-openvswitch\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.077147 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.076863 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-host-cni-netd\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.077147 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.076883 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dcc18d79-d518-4d89-ac61-0e3e1ff55794-system-cni-dir\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.077147 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.076904 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dcc18d79-d518-4d89-ac61-0e3e1ff55794-host-run-k8s-cni-cncf-io\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.077147 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.076933 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dcc18d79-d518-4d89-ac61-0e3e1ff55794-host-var-lib-cni-multus\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.077147 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.077017 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dcc18d79-d518-4d89-ac61-0e3e1ff55794-multus-daemon-config\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.077147 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.077064 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dcc18d79-d518-4d89-ac61-0e3e1ff55794-etc-kubernetes\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.077147 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.077097 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dcc18d79-d518-4d89-ac61-0e3e1ff55794-host-run-netns\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.077147 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.077117 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dcc18d79-d518-4d89-ac61-0e3e1ff55794-hostroot\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.077866 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.077140 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ftsb\" (UniqueName: \"kubernetes.io/projected/dcc18d79-d518-4d89-ac61-0e3e1ff55794-kube-api-access-2ftsb\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.077866 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.077159 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0fe9b158-c1b2-498d-aecc-d32aee3c51c6-host\") pod \"node-ca-jx58q\" (UID: \"0fe9b158-c1b2-498d-aecc-d32aee3c51c6\") " pod="openshift-image-registry/node-ca-jx58q" Sep 09 22:59:01.077866 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.077185 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0fe9b158-c1b2-498d-aecc-d32aee3c51c6-serviceca\") pod \"node-ca-jx58q\" (UID: \"0fe9b158-c1b2-498d-aecc-d32aee3c51c6\") " pod="openshift-image-registry/node-ca-jx58q" Sep 09 22:59:01.077866 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.077222 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6csbz\" (UniqueName: \"kubernetes.io/projected/0fe9b158-c1b2-498d-aecc-d32aee3c51c6-kube-api-access-6csbz\") pod \"node-ca-jx58q\" (UID: \"0fe9b158-c1b2-498d-aecc-d32aee3c51c6\") " pod="openshift-image-registry/node-ca-jx58q" Sep 09 22:59:01.077866 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.077245 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dcc18d79-d518-4d89-ac61-0e3e1ff55794-host-var-lib-cni-bin\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.077866 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.077271 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/09c46a31-75e9-40a5-856f-4f53ae4b0a3b-system-cni-dir\") pod \"multus-additional-cni-plugins-dc58c\" (UID: \"09c46a31-75e9-40a5-856f-4f53ae4b0a3b\") " pod="openshift-multus/multus-additional-cni-plugins-dc58c" Sep 09 22:59:01.077866 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.077293 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cbe663c-0531-4339-b48b-c4d4c3dd5c2b-metrics-certs\") pod \"network-metrics-daemon-8dwkd\" (UID: \"7cbe663c-0531-4339-b48b-c4d4c3dd5c2b\") " pod="openshift-multus/network-metrics-daemon-8dwkd" Sep 09 22:59:01.077866 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.077313 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dadd077d-e65f-49c0-a1bf-b8299ef2bb89-hosts-file\") pod \"node-resolver-lwqr2\" (UID: \"dadd077d-e65f-49c0-a1bf-b8299ef2bb89\") " pod="openshift-dns/node-resolver-lwqr2" Sep 09 22:59:01.077866 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.077335 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dadd077d-e65f-49c0-a1bf-b8299ef2bb89-tmp-dir\") pod \"node-resolver-lwqr2\" (UID: \"dadd077d-e65f-49c0-a1bf-b8299ef2bb89\") " pod="openshift-dns/node-resolver-lwqr2" Sep 09 22:59:01.077866 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.077356 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-host-slash\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.077866 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.077383 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-run-openvswitch\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.077866 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.077404 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dcc18d79-d518-4d89-ac61-0e3e1ff55794-multus-cni-dir\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.077866 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.077431 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2a2906d3-5dc1-441e-b7f8-a97e2a67d4ed-konnectivity-ca\") pod \"konnectivity-agent-ps2b2\" (UID: \"2a2906d3-5dc1-441e-b7f8-a97e2a67d4ed\") " pod="kube-system/konnectivity-agent-ps2b2" Sep 09 22:59:01.077866 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.077453 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/09c46a31-75e9-40a5-856f-4f53ae4b0a3b-cnibin\") pod \"multus-additional-cni-plugins-dc58c\" (UID: \"09c46a31-75e9-40a5-856f-4f53ae4b0a3b\") " pod="openshift-multus/multus-additional-cni-plugins-dc58c" Sep 09 22:59:01.077866 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.077475 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/09c46a31-75e9-40a5-856f-4f53ae4b0a3b-cni-binary-copy\") pod \"multus-additional-cni-plugins-dc58c\" (UID: \"09c46a31-75e9-40a5-856f-4f53ae4b0a3b\") " pod="openshift-multus/multus-additional-cni-plugins-dc58c" Sep 09 22:59:01.077866 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.077504 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/09c46a31-75e9-40a5-856f-4f53ae4b0a3b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dc58c\" (UID: \"09c46a31-75e9-40a5-856f-4f53ae4b0a3b\") " pod="openshift-multus/multus-additional-cni-plugins-dc58c" Sep 09 22:59:01.078566 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.077531 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/09c46a31-75e9-40a5-856f-4f53ae4b0a3b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dc58c\" (UID: \"09c46a31-75e9-40a5-856f-4f53ae4b0a3b\") " pod="openshift-multus/multus-additional-cni-plugins-dc58c" Sep 09 22:59:01.078566 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.077554 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-systemd-units\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.078566 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.077583 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-host-run-netns\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.078566 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.077585 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-h25jx\"" Sep 09 22:59:01.078566 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.077604 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-run-ovn\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.078566 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.077627 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-node-log\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.078566 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.077814 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 09 22:59:01.078566 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.078207 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 09 22:59:01.078930 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.078874 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gdhb9" Sep 09 22:59:01.079049 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:01.078940 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gdhb9" podUID="b85f7fcb-c8f2-43aa-9d28-890c87afcdd0" Sep 09 22:59:01.081533 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.081516 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7mz5t" Sep 09 22:59:01.083288 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.083271 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2gxt" Sep 09 22:59:01.083357 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:01.083325 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2gxt" podUID="66786687-e3fe-42fd-8199-7f2a95232aa8" Sep 09 22:59:01.084272 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.084253 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 09 22:59:01.084716 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.084700 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-5w6nn\"" Sep 09 22:59:01.084813 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.084797 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 09 22:59:01.085777 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.085762 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 09 22:59:01.086402 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.086383 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 09 22:59:01.119352 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.119326 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-2d85k" Sep 09 22:59:01.126950 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.126923 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2d85k" Sep 09 22:59:01.167447 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.167418 2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 09 22:59:01.178471 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.178430 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-host-kubelet\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.178629 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.178491 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvjcm\" (UniqueName: \"kubernetes.io/projected/bece1799-8601-448b-9fb5-67fe028e3d7e-kube-api-access-xvjcm\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.178629 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.178518 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2a2906d3-5dc1-441e-b7f8-a97e2a67d4ed-agent-certs\") pod \"konnectivity-agent-ps2b2\" (UID: \"2a2906d3-5dc1-441e-b7f8-a97e2a67d4ed\") " pod="kube-system/konnectivity-agent-ps2b2" Sep 09 22:59:01.178629 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.178550 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/785b3628-74d5-4a5a-b3c5-10b7d2c72140-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5pckv\" (UID: \"785b3628-74d5-4a5a-b3c5-10b7d2c72140\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pckv" Sep 09 22:59:01.178629 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.178578 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-host-kubelet\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.178629 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.178581 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a3574ed6-8639-4a22-b118-ebb5500b086d-etc-systemd\") pod \"tuned-cb4k2\" (UID: \"a3574ed6-8639-4a22-b118-ebb5500b086d\") " pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.178629 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.178625 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a3574ed6-8639-4a22-b118-ebb5500b086d-lib-modules\") pod \"tuned-cb4k2\" (UID: \"a3574ed6-8639-4a22-b118-ebb5500b086d\") " pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.179084 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.178653 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a3574ed6-8639-4a22-b118-ebb5500b086d-host\") pod \"tuned-cb4k2\" (UID: \"a3574ed6-8639-4a22-b118-ebb5500b086d\") " pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.179084 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.178677 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/66786687-e3fe-42fd-8199-7f2a95232aa8-dbus\") pod \"global-pull-secret-syncer-b2gxt\" (UID: \"66786687-e3fe-42fd-8199-7f2a95232aa8\") " pod="kube-system/global-pull-secret-syncer-b2gxt" Sep 09 22:59:01.179084 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.178712 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/09c46a31-75e9-40a5-856f-4f53ae4b0a3b-cnibin\") pod \"multus-additional-cni-plugins-dc58c\" (UID: \"09c46a31-75e9-40a5-856f-4f53ae4b0a3b\") " pod="openshift-multus/multus-additional-cni-plugins-dc58c" Sep 09 22:59:01.179084 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.178729 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/09c46a31-75e9-40a5-856f-4f53ae4b0a3b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dc58c\" (UID: \"09c46a31-75e9-40a5-856f-4f53ae4b0a3b\") " pod="openshift-multus/multus-additional-cni-plugins-dc58c" Sep 09 22:59:01.179084 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.178746 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dcc18d79-d518-4d89-ac61-0e3e1ff55794-cnibin\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.179084 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.178760 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-host-cni-netd\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.179084 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.178786 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dcc18d79-d518-4d89-ac61-0e3e1ff55794-host-run-k8s-cni-cncf-io\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.179084 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.178809 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dcc18d79-d518-4d89-ac61-0e3e1ff55794-host-var-lib-cni-multus\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.179084 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.178843 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dcc18d79-d518-4d89-ac61-0e3e1ff55794-etc-kubernetes\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.179084 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.178875 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/785b3628-74d5-4a5a-b3c5-10b7d2c72140-socket-dir\") pod \"aws-ebs-csi-driver-node-5pckv\" (UID: \"785b3628-74d5-4a5a-b3c5-10b7d2c72140\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pckv" Sep 09 22:59:01.179549 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.179211 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dcc18d79-d518-4d89-ac61-0e3e1ff55794-cnibin\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.179549 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.179273 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dcc18d79-d518-4d89-ac61-0e3e1ff55794-host-var-lib-cni-multus\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.179549 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.179363 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-host-cni-netd\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.179549 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.179363 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/09c46a31-75e9-40a5-856f-4f53ae4b0a3b-cnibin\") pod \"multus-additional-cni-plugins-dc58c\" (UID: \"09c46a31-75e9-40a5-856f-4f53ae4b0a3b\") " pod="openshift-multus/multus-additional-cni-plugins-dc58c" Sep 09 22:59:01.179549 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.179423 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dcc18d79-d518-4d89-ac61-0e3e1ff55794-host-run-netns\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.179549 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.179438 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dcc18d79-d518-4d89-ac61-0e3e1ff55794-etc-kubernetes\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.179549 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.179460 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dcc18d79-d518-4d89-ac61-0e3e1ff55794-hostroot\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.179549 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.179479 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dcc18d79-d518-4d89-ac61-0e3e1ff55794-host-run-netns\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.179549 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.179494 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/09c46a31-75e9-40a5-856f-4f53ae4b0a3b-system-cni-dir\") pod \"multus-additional-cni-plugins-dc58c\" (UID: \"09c46a31-75e9-40a5-856f-4f53ae4b0a3b\") " pod="openshift-multus/multus-additional-cni-plugins-dc58c" Sep 09 22:59:01.179549 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.179530 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dcc18d79-d518-4d89-ac61-0e3e1ff55794-hostroot\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.179549 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.179530 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dadd077d-e65f-49c0-a1bf-b8299ef2bb89-hosts-file\") pod \"node-resolver-lwqr2\" (UID: \"dadd077d-e65f-49c0-a1bf-b8299ef2bb89\") " pod="openshift-dns/node-resolver-lwqr2" Sep 09 22:59:01.180035 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.179577 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dadd077d-e65f-49c0-a1bf-b8299ef2bb89-hosts-file\") pod \"node-resolver-lwqr2\" (UID: \"dadd077d-e65f-49c0-a1bf-b8299ef2bb89\") " pod="openshift-dns/node-resolver-lwqr2" Sep 09 22:59:01.180035 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.179600 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/09c46a31-75e9-40a5-856f-4f53ae4b0a3b-system-cni-dir\") pod \"multus-additional-cni-plugins-dc58c\" (UID: \"09c46a31-75e9-40a5-856f-4f53ae4b0a3b\") " pod="openshift-multus/multus-additional-cni-plugins-dc58c" Sep 09 22:59:01.180035 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.179639 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cbe663c-0531-4339-b48b-c4d4c3dd5c2b-metrics-certs\") pod \"network-metrics-daemon-8dwkd\" (UID: \"7cbe663c-0531-4339-b48b-c4d4c3dd5c2b\") " pod="openshift-multus/network-metrics-daemon-8dwkd" Sep 09 22:59:01.180035 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.179666 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-host-slash\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.180035 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.179684 2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 09 22:59:01.180035 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.179755 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-host-slash\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.180035 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:01.179833 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 22:59:01.180035 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:01.179949 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cbe663c-0531-4339-b48b-c4d4c3dd5c2b-metrics-certs podName:7cbe663c-0531-4339-b48b-c4d4c3dd5c2b nodeName:}" failed. No retries permitted until 2026-09-09 22:59:01.679877556 +0000 UTC m=+3.029744572 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7cbe663c-0531-4339-b48b-c4d4c3dd5c2b-metrics-certs") pod "network-metrics-daemon-8dwkd" (UID: "7cbe663c-0531-4339-b48b-c4d4c3dd5c2b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 22:59:01.180035 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.180001 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dcc18d79-d518-4d89-ac61-0e3e1ff55794-multus-cni-dir\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.180035 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.180034 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a3574ed6-8639-4a22-b118-ebb5500b086d-etc-sysctl-conf\") pod \"tuned-cb4k2\" (UID: \"a3574ed6-8639-4a22-b118-ebb5500b086d\") " pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.180432 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.180070 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3574ed6-8639-4a22-b118-ebb5500b086d-var-lib-kubelet\") pod \"tuned-cb4k2\" (UID: \"a3574ed6-8639-4a22-b118-ebb5500b086d\") " pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.180432 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.180100 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-systemd-units\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.180432 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.180128 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bece1799-8601-448b-9fb5-67fe028e3d7e-ovn-node-metrics-cert\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.180432 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.180156 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/66786687-e3fe-42fd-8199-7f2a95232aa8-kubelet-config\") pod \"global-pull-secret-syncer-b2gxt\" (UID: \"66786687-e3fe-42fd-8199-7f2a95232aa8\") " pod="kube-system/global-pull-secret-syncer-b2gxt" Sep 09 22:59:01.180432 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.180251 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-etc-openvswitch\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.180432 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.180302 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dcc18d79-d518-4d89-ac61-0e3e1ff55794-host-run-multus-certs\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.180432 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.180328 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghdz7\" (UniqueName: \"kubernetes.io/projected/09c46a31-75e9-40a5-856f-4f53ae4b0a3b-kube-api-access-ghdz7\") pod \"multus-additional-cni-plugins-dc58c\" (UID: \"09c46a31-75e9-40a5-856f-4f53ae4b0a3b\") " pod="openshift-multus/multus-additional-cni-plugins-dc58c" Sep 09 22:59:01.180432 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.180357 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a3574ed6-8639-4a22-b118-ebb5500b086d-etc-modprobe-d\") pod \"tuned-cb4k2\" (UID: \"a3574ed6-8639-4a22-b118-ebb5500b086d\") " pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.180786 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.180529 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/66786687-e3fe-42fd-8199-7f2a95232aa8-original-pull-secret\") pod \"global-pull-secret-syncer-b2gxt\" (UID: \"66786687-e3fe-42fd-8199-7f2a95232aa8\") " pod="kube-system/global-pull-secret-syncer-b2gxt" Sep 09 22:59:01.180786 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.180559 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dcc18d79-d518-4d89-ac61-0e3e1ff55794-multus-socket-dir-parent\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.180786 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.180595 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a3574ed6-8639-4a22-b118-ebb5500b086d-etc-tuned\") pod \"tuned-cb4k2\" (UID: \"a3574ed6-8639-4a22-b118-ebb5500b086d\") " pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.180786 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.180639 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b43af431-82d9-45b1-937a-5c1cede57ac2-host-slash\") pod \"iptables-alerter-7mz5t\" (UID: \"b43af431-82d9-45b1-937a-5c1cede57ac2\") " pod="openshift-network-operator/iptables-alerter-7mz5t" Sep 09 22:59:01.180786 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.180676 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-run-systemd\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.180786 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.180715 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.180786 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.180753 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dcc18d79-d518-4d89-ac61-0e3e1ff55794-cni-binary-copy\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.180786 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.180778 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wg4w9\" (UniqueName: \"kubernetes.io/projected/dadd077d-e65f-49c0-a1bf-b8299ef2bb89-kube-api-access-wg4w9\") pod \"node-resolver-lwqr2\" (UID: \"dadd077d-e65f-49c0-a1bf-b8299ef2bb89\") " pod="openshift-dns/node-resolver-lwqr2" Sep 09 22:59:01.185082 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.181746 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.185082 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.181816 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dcc18d79-d518-4d89-ac61-0e3e1ff55794-host-run-multus-certs\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.185082 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.181858 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/785b3628-74d5-4a5a-b3c5-10b7d2c72140-etc-selinux\") pod \"aws-ebs-csi-driver-node-5pckv\" (UID: \"785b3628-74d5-4a5a-b3c5-10b7d2c72140\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pckv" Sep 09 22:59:01.185082 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.181884 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cv2ml\" (UniqueName: \"kubernetes.io/projected/785b3628-74d5-4a5a-b3c5-10b7d2c72140-kube-api-access-cv2ml\") pod \"aws-ebs-csi-driver-node-5pckv\" (UID: \"785b3628-74d5-4a5a-b3c5-10b7d2c72140\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pckv" Sep 09 22:59:01.185082 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.181919 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a3574ed6-8639-4a22-b118-ebb5500b086d-etc-sysconfig\") pod \"tuned-cb4k2\" (UID: \"a3574ed6-8639-4a22-b118-ebb5500b086d\") " pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.185082 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.181948 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0fe9b158-c1b2-498d-aecc-d32aee3c51c6-host\") pod \"node-ca-jx58q\" (UID: \"0fe9b158-c1b2-498d-aecc-d32aee3c51c6\") " pod="openshift-image-registry/node-ca-jx58q" Sep 09 22:59:01.185082 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.181992 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0fe9b158-c1b2-498d-aecc-d32aee3c51c6-serviceca\") pod \"node-ca-jx58q\" (UID: \"0fe9b158-c1b2-498d-aecc-d32aee3c51c6\") " pod="openshift-image-registry/node-ca-jx58q" Sep 09 22:59:01.185082 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.182015 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-var-lib-openvswitch\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.185082 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.182055 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dcc18d79-d518-4d89-ac61-0e3e1ff55794-system-cni-dir\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.185082 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.182093 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dcc18d79-d518-4d89-ac61-0e3e1ff55794-multus-daemon-config\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.185082 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.182120 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pd27g\" (UniqueName: \"kubernetes.io/projected/b85f7fcb-c8f2-43aa-9d28-890c87afcdd0-kube-api-access-pd27g\") pod \"network-check-target-gdhb9\" (UID: \"b85f7fcb-c8f2-43aa-9d28-890c87afcdd0\") " pod="openshift-network-diagnostics/network-check-target-gdhb9" Sep 09 22:59:01.185082 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.182162 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ftsb\" (UniqueName: \"kubernetes.io/projected/dcc18d79-d518-4d89-ac61-0e3e1ff55794-kube-api-access-2ftsb\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.185082 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.182315 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dcc18d79-d518-4d89-ac61-0e3e1ff55794-host-run-k8s-cni-cncf-io\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.185082 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.182514 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-run-systemd\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.185082 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.182615 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dcc18d79-d518-4d89-ac61-0e3e1ff55794-system-cni-dir\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.185082 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.182769 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0fe9b158-c1b2-498d-aecc-d32aee3c51c6-host\") pod \"node-ca-jx58q\" (UID: \"0fe9b158-c1b2-498d-aecc-d32aee3c51c6\") " pod="openshift-image-registry/node-ca-jx58q" Sep 09 22:59:01.185082 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.182800 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-92.ec2.internal" event={"ID":"8d896ae2507ca9bca5f4fea55fdadcb0","Type":"ContainerStarted","Data":"21fde6e75ff814f098de13b0b77bce64310c1784f96e6bff4d6b372a8c6c5a93"} Sep 09 22:59:01.185931 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.182945 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-etc-openvswitch\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.185931 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.183004 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dcc18d79-d518-4d89-ac61-0e3e1ff55794-cni-binary-copy\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.185931 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.183002 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dcc18d79-d518-4d89-ac61-0e3e1ff55794-multus-cni-dir\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.185931 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.183080 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-systemd-units\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.185931 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.183079 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0fe9b158-c1b2-498d-aecc-d32aee3c51c6-serviceca\") pod \"node-ca-jx58q\" (UID: \"0fe9b158-c1b2-498d-aecc-d32aee3c51c6\") " pod="openshift-image-registry/node-ca-jx58q" Sep 09 22:59:01.185931 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.183088 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6csbz\" (UniqueName: \"kubernetes.io/projected/0fe9b158-c1b2-498d-aecc-d32aee3c51c6-kube-api-access-6csbz\") pod \"node-ca-jx58q\" (UID: \"0fe9b158-c1b2-498d-aecc-d32aee3c51c6\") " pod="openshift-image-registry/node-ca-jx58q" Sep 09 22:59:01.185931 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.183121 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-var-lib-openvswitch\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.185931 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.183135 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/785b3628-74d5-4a5a-b3c5-10b7d2c72140-device-dir\") pod \"aws-ebs-csi-driver-node-5pckv\" (UID: \"785b3628-74d5-4a5a-b3c5-10b7d2c72140\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pckv" Sep 09 22:59:01.185931 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.183170 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dcc18d79-d518-4d89-ac61-0e3e1ff55794-multus-socket-dir-parent\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.185931 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.183218 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/785b3628-74d5-4a5a-b3c5-10b7d2c72140-sys-fs\") pod \"aws-ebs-csi-driver-node-5pckv\" (UID: \"785b3628-74d5-4a5a-b3c5-10b7d2c72140\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pckv" Sep 09 22:59:01.185931 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.183255 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3574ed6-8639-4a22-b118-ebb5500b086d-etc-kubernetes\") pod \"tuned-cb4k2\" (UID: \"a3574ed6-8639-4a22-b118-ebb5500b086d\") " pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.185931 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.183338 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a3574ed6-8639-4a22-b118-ebb5500b086d-run\") pod \"tuned-cb4k2\" (UID: \"a3574ed6-8639-4a22-b118-ebb5500b086d\") " pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.185931 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.183415 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b43af431-82d9-45b1-937a-5c1cede57ac2-iptables-alerter-script\") pod \"iptables-alerter-7mz5t\" (UID: \"b43af431-82d9-45b1-937a-5c1cede57ac2\") " pod="openshift-network-operator/iptables-alerter-7mz5t" Sep 09 22:59:01.185931 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.183442 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dcc18d79-d518-4d89-ac61-0e3e1ff55794-multus-daemon-config\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.185931 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.183501 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dcc18d79-d518-4d89-ac61-0e3e1ff55794-host-var-lib-cni-bin\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.185931 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.183579 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dadd077d-e65f-49c0-a1bf-b8299ef2bb89-tmp-dir\") pod \"node-resolver-lwqr2\" (UID: \"dadd077d-e65f-49c0-a1bf-b8299ef2bb89\") " pod="openshift-dns/node-resolver-lwqr2" Sep 09 22:59:01.185931 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.183718 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a3574ed6-8639-4a22-b118-ebb5500b086d-tmp\") pod \"tuned-cb4k2\" (UID: \"a3574ed6-8639-4a22-b118-ebb5500b086d\") " pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.186709 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.183739 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-run-openvswitch\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.186709 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.183919 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2a2906d3-5dc1-441e-b7f8-a97e2a67d4ed-konnectivity-ca\") pod \"konnectivity-agent-ps2b2\" (UID: \"2a2906d3-5dc1-441e-b7f8-a97e2a67d4ed\") " pod="kube-system/konnectivity-agent-ps2b2" Sep 09 22:59:01.186709 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.183948 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/09c46a31-75e9-40a5-856f-4f53ae4b0a3b-cni-binary-copy\") pod \"multus-additional-cni-plugins-dc58c\" (UID: \"09c46a31-75e9-40a5-856f-4f53ae4b0a3b\") " pod="openshift-multus/multus-additional-cni-plugins-dc58c" Sep 09 22:59:01.186709 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.183992 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/09c46a31-75e9-40a5-856f-4f53ae4b0a3b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dc58c\" (UID: \"09c46a31-75e9-40a5-856f-4f53ae4b0a3b\") " pod="openshift-multus/multus-additional-cni-plugins-dc58c" Sep 09 22:59:01.186709 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.184011 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dadd077d-e65f-49c0-a1bf-b8299ef2bb89-tmp-dir\") pod \"node-resolver-lwqr2\" (UID: \"dadd077d-e65f-49c0-a1bf-b8299ef2bb89\") " pod="openshift-dns/node-resolver-lwqr2" Sep 09 22:59:01.186709 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.183623 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dcc18d79-d518-4d89-ac61-0e3e1ff55794-host-var-lib-cni-bin\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.186709 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.184085 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-run-openvswitch\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.186709 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.184250 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/09c46a31-75e9-40a5-856f-4f53ae4b0a3b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dc58c\" (UID: \"09c46a31-75e9-40a5-856f-4f53ae4b0a3b\") " pod="openshift-multus/multus-additional-cni-plugins-dc58c" Sep 09 22:59:01.186709 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.184272 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-host-run-netns\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.186709 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.184392 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-host-run-netns\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.186709 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.184741 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2a2906d3-5dc1-441e-b7f8-a97e2a67d4ed-konnectivity-ca\") pod \"konnectivity-agent-ps2b2\" (UID: \"2a2906d3-5dc1-441e-b7f8-a97e2a67d4ed\") " pod="kube-system/konnectivity-agent-ps2b2" Sep 09 22:59:01.186709 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.184852 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/09c46a31-75e9-40a5-856f-4f53ae4b0a3b-cni-binary-copy\") pod \"multus-additional-cni-plugins-dc58c\" (UID: \"09c46a31-75e9-40a5-856f-4f53ae4b0a3b\") " pod="openshift-multus/multus-additional-cni-plugins-dc58c" Sep 09 22:59:01.186709 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.184877 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-run-ovn\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.186709 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.184346 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-run-ovn\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.186709 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.185096 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-node-log\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.186709 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.185140 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-node-log\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.186709 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.185172 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dcc18d79-d518-4d89-ac61-0e3e1ff55794-multus-conf-dir\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.187540 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.185202 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/09c46a31-75e9-40a5-856f-4f53ae4b0a3b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dc58c\" (UID: \"09c46a31-75e9-40a5-856f-4f53ae4b0a3b\") " pod="openshift-multus/multus-additional-cni-plugins-dc58c" Sep 09 22:59:01.187540 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.185216 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/09c46a31-75e9-40a5-856f-4f53ae4b0a3b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dc58c\" (UID: \"09c46a31-75e9-40a5-856f-4f53ae4b0a3b\") " pod="openshift-multus/multus-additional-cni-plugins-dc58c" Sep 09 22:59:01.187540 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.185224 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xttlb\" (UniqueName: \"kubernetes.io/projected/7cbe663c-0531-4339-b48b-c4d4c3dd5c2b-kube-api-access-xttlb\") pod \"network-metrics-daemon-8dwkd\" (UID: \"7cbe663c-0531-4339-b48b-c4d4c3dd5c2b\") " pod="openshift-multus/network-metrics-daemon-8dwkd" Sep 09 22:59:01.187540 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.185253 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a3574ed6-8639-4a22-b118-ebb5500b086d-etc-sysctl-d\") pod \"tuned-cb4k2\" (UID: \"a3574ed6-8639-4a22-b118-ebb5500b086d\") " pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.187540 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.185276 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dcc18d79-d518-4d89-ac61-0e3e1ff55794-multus-conf-dir\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.187540 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.185286 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dcc18d79-d518-4d89-ac61-0e3e1ff55794-host-var-lib-kubelet\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.187540 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.185332 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dcc18d79-d518-4d89-ac61-0e3e1ff55794-host-var-lib-kubelet\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.187540 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.185344 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/785b3628-74d5-4a5a-b3c5-10b7d2c72140-registration-dir\") pod \"aws-ebs-csi-driver-node-5pckv\" (UID: \"785b3628-74d5-4a5a-b3c5-10b7d2c72140\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pckv" Sep 09 22:59:01.187540 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.185380 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a3574ed6-8639-4a22-b118-ebb5500b086d-sys\") pod \"tuned-cb4k2\" (UID: \"a3574ed6-8639-4a22-b118-ebb5500b086d\") " pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.187540 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.185404 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpwqw\" (UniqueName: \"kubernetes.io/projected/a3574ed6-8639-4a22-b118-ebb5500b086d-kube-api-access-xpwqw\") pod \"tuned-cb4k2\" (UID: \"a3574ed6-8639-4a22-b118-ebb5500b086d\") " pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.187540 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.185432 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-log-socket\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.187540 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.185459 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-host-run-ovn-kubernetes\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.187540 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.185494 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-host-cni-bin\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.187540 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.185517 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bece1799-8601-448b-9fb5-67fe028e3d7e-ovnkube-config\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.187540 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.185543 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bece1799-8601-448b-9fb5-67fe028e3d7e-env-overrides\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.187540 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.185588 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bece1799-8601-448b-9fb5-67fe028e3d7e-ovnkube-script-lib\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.187540 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.185619 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-log-socket\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.188309 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.185634 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/09c46a31-75e9-40a5-856f-4f53ae4b0a3b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dc58c\" (UID: \"09c46a31-75e9-40a5-856f-4f53ae4b0a3b\") " pod="openshift-multus/multus-additional-cni-plugins-dc58c" Sep 09 22:59:01.188309 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.185649 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kr95\" (UniqueName: \"kubernetes.io/projected/b43af431-82d9-45b1-937a-5c1cede57ac2-kube-api-access-4kr95\") pod \"iptables-alerter-7mz5t\" (UID: \"b43af431-82d9-45b1-937a-5c1cede57ac2\") " pod="openshift-network-operator/iptables-alerter-7mz5t" Sep 09 22:59:01.188309 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.185807 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-host-cni-bin\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.188309 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.185858 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bece1799-8601-448b-9fb5-67fe028e3d7e-host-run-ovn-kubernetes\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.188309 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.186268 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bece1799-8601-448b-9fb5-67fe028e3d7e-env-overrides\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.188309 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.186303 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bece1799-8601-448b-9fb5-67fe028e3d7e-ovnkube-config\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.188309 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.187057 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bece1799-8601-448b-9fb5-67fe028e3d7e-ovnkube-script-lib\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.188309 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.187331 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bece1799-8601-448b-9fb5-67fe028e3d7e-ovn-node-metrics-cert\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.191286 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.191251 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvjcm\" (UniqueName: \"kubernetes.io/projected/bece1799-8601-448b-9fb5-67fe028e3d7e-kube-api-access-xvjcm\") pod \"ovnkube-node-prbnz\" (UID: \"bece1799-8601-448b-9fb5-67fe028e3d7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.191393 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.191300 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wg4w9\" (UniqueName: \"kubernetes.io/projected/dadd077d-e65f-49c0-a1bf-b8299ef2bb89-kube-api-access-wg4w9\") pod \"node-resolver-lwqr2\" (UID: \"dadd077d-e65f-49c0-a1bf-b8299ef2bb89\") " pod="openshift-dns/node-resolver-lwqr2" Sep 09 22:59:01.191531 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.191510 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ftsb\" (UniqueName: \"kubernetes.io/projected/dcc18d79-d518-4d89-ac61-0e3e1ff55794-kube-api-access-2ftsb\") pod \"multus-w9lv9\" (UID: \"dcc18d79-d518-4d89-ac61-0e3e1ff55794\") " pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.191657 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.191636 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2a2906d3-5dc1-441e-b7f8-a97e2a67d4ed-agent-certs\") pod \"konnectivity-agent-ps2b2\" (UID: \"2a2906d3-5dc1-441e-b7f8-a97e2a67d4ed\") " pod="kube-system/konnectivity-agent-ps2b2" Sep 09 22:59:01.191726 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.191649 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-92.ec2.internal" event={"ID":"fa1cb3f75de266c7e0e4d8ba90349b01","Type":"ContainerStarted","Data":"ef52789e3b7baf25762a03462e08effc3bb27b06aa786b4c6df89788dbd0ce28"} Sep 09 22:59:01.194262 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.194239 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xttlb\" (UniqueName: \"kubernetes.io/projected/7cbe663c-0531-4339-b48b-c4d4c3dd5c2b-kube-api-access-xttlb\") pod \"network-metrics-daemon-8dwkd\" (UID: \"7cbe663c-0531-4339-b48b-c4d4c3dd5c2b\") " pod="openshift-multus/network-metrics-daemon-8dwkd" Sep 09 22:59:01.197255 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.196898 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghdz7\" (UniqueName: \"kubernetes.io/projected/09c46a31-75e9-40a5-856f-4f53ae4b0a3b-kube-api-access-ghdz7\") pod \"multus-additional-cni-plugins-dc58c\" (UID: \"09c46a31-75e9-40a5-856f-4f53ae4b0a3b\") " pod="openshift-multus/multus-additional-cni-plugins-dc58c" Sep 09 22:59:01.197508 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.197491 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6csbz\" (UniqueName: \"kubernetes.io/projected/0fe9b158-c1b2-498d-aecc-d32aee3c51c6-kube-api-access-6csbz\") pod \"node-ca-jx58q\" (UID: \"0fe9b158-c1b2-498d-aecc-d32aee3c51c6\") " pod="openshift-image-registry/node-ca-jx58q" Sep 09 22:59:01.286749 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.286715 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/785b3628-74d5-4a5a-b3c5-10b7d2c72140-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5pckv\" (UID: \"785b3628-74d5-4a5a-b3c5-10b7d2c72140\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pckv" Sep 09 22:59:01.286882 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.286759 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a3574ed6-8639-4a22-b118-ebb5500b086d-etc-systemd\") pod \"tuned-cb4k2\" (UID: \"a3574ed6-8639-4a22-b118-ebb5500b086d\") " pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.286882 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.286781 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a3574ed6-8639-4a22-b118-ebb5500b086d-lib-modules\") pod \"tuned-cb4k2\" (UID: \"a3574ed6-8639-4a22-b118-ebb5500b086d\") " pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.286882 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.286855 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a3574ed6-8639-4a22-b118-ebb5500b086d-etc-systemd\") pod \"tuned-cb4k2\" (UID: \"a3574ed6-8639-4a22-b118-ebb5500b086d\") " pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.287075 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.286914 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/785b3628-74d5-4a5a-b3c5-10b7d2c72140-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5pckv\" (UID: \"785b3628-74d5-4a5a-b3c5-10b7d2c72140\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pckv" Sep 09 22:59:01.287075 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.286938 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a3574ed6-8639-4a22-b118-ebb5500b086d-host\") pod \"tuned-cb4k2\" (UID: \"a3574ed6-8639-4a22-b118-ebb5500b086d\") " pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.287075 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.286951 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/66786687-e3fe-42fd-8199-7f2a95232aa8-dbus\") pod \"global-pull-secret-syncer-b2gxt\" (UID: \"66786687-e3fe-42fd-8199-7f2a95232aa8\") " pod="kube-system/global-pull-secret-syncer-b2gxt" Sep 09 22:59:01.287075 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.286996 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/785b3628-74d5-4a5a-b3c5-10b7d2c72140-socket-dir\") pod \"aws-ebs-csi-driver-node-5pckv\" (UID: \"785b3628-74d5-4a5a-b3c5-10b7d2c72140\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pckv" Sep 09 22:59:01.287075 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.287034 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a3574ed6-8639-4a22-b118-ebb5500b086d-etc-sysctl-conf\") pod \"tuned-cb4k2\" (UID: \"a3574ed6-8639-4a22-b118-ebb5500b086d\") " pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.287075 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.287047 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a3574ed6-8639-4a22-b118-ebb5500b086d-host\") pod \"tuned-cb4k2\" (UID: \"a3574ed6-8639-4a22-b118-ebb5500b086d\") " pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.287075 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.287058 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a3574ed6-8639-4a22-b118-ebb5500b086d-lib-modules\") pod \"tuned-cb4k2\" (UID: \"a3574ed6-8639-4a22-b118-ebb5500b086d\") " pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.287075 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.287055 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3574ed6-8639-4a22-b118-ebb5500b086d-var-lib-kubelet\") pod \"tuned-cb4k2\" (UID: \"a3574ed6-8639-4a22-b118-ebb5500b086d\") " pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.287420 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.287088 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3574ed6-8639-4a22-b118-ebb5500b086d-var-lib-kubelet\") pod \"tuned-cb4k2\" (UID: \"a3574ed6-8639-4a22-b118-ebb5500b086d\") " pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.287420 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.287105 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/66786687-e3fe-42fd-8199-7f2a95232aa8-kubelet-config\") pod \"global-pull-secret-syncer-b2gxt\" (UID: \"66786687-e3fe-42fd-8199-7f2a95232aa8\") " pod="kube-system/global-pull-secret-syncer-b2gxt" Sep 09 22:59:01.287420 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.287149 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a3574ed6-8639-4a22-b118-ebb5500b086d-etc-modprobe-d\") pod \"tuned-cb4k2\" (UID: \"a3574ed6-8639-4a22-b118-ebb5500b086d\") " pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.287420 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.287173 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/66786687-e3fe-42fd-8199-7f2a95232aa8-original-pull-secret\") pod \"global-pull-secret-syncer-b2gxt\" (UID: \"66786687-e3fe-42fd-8199-7f2a95232aa8\") " pod="kube-system/global-pull-secret-syncer-b2gxt" Sep 09 22:59:01.287420 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.287186 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a3574ed6-8639-4a22-b118-ebb5500b086d-etc-sysctl-conf\") pod \"tuned-cb4k2\" (UID: \"a3574ed6-8639-4a22-b118-ebb5500b086d\") " pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.287420 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.287163 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/785b3628-74d5-4a5a-b3c5-10b7d2c72140-socket-dir\") pod \"aws-ebs-csi-driver-node-5pckv\" (UID: \"785b3628-74d5-4a5a-b3c5-10b7d2c72140\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pckv" Sep 09 22:59:01.287420 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.287200 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a3574ed6-8639-4a22-b118-ebb5500b086d-etc-tuned\") pod \"tuned-cb4k2\" (UID: \"a3574ed6-8639-4a22-b118-ebb5500b086d\") " pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.287420 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.287228 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b43af431-82d9-45b1-937a-5c1cede57ac2-host-slash\") pod \"iptables-alerter-7mz5t\" (UID: \"b43af431-82d9-45b1-937a-5c1cede57ac2\") " pod="openshift-network-operator/iptables-alerter-7mz5t" Sep 09 22:59:01.287420 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.287252 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/785b3628-74d5-4a5a-b3c5-10b7d2c72140-etc-selinux\") pod \"aws-ebs-csi-driver-node-5pckv\" (UID: \"785b3628-74d5-4a5a-b3c5-10b7d2c72140\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pckv" Sep 09 22:59:01.287420 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.287261 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a3574ed6-8639-4a22-b118-ebb5500b086d-etc-modprobe-d\") pod \"tuned-cb4k2\" (UID: \"a3574ed6-8639-4a22-b118-ebb5500b086d\") " pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.287420 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.287275 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cv2ml\" (UniqueName: \"kubernetes.io/projected/785b3628-74d5-4a5a-b3c5-10b7d2c72140-kube-api-access-cv2ml\") pod \"aws-ebs-csi-driver-node-5pckv\" (UID: \"785b3628-74d5-4a5a-b3c5-10b7d2c72140\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pckv" Sep 09 22:59:01.287420 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.287302 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/66786687-e3fe-42fd-8199-7f2a95232aa8-kubelet-config\") pod \"global-pull-secret-syncer-b2gxt\" (UID: \"66786687-e3fe-42fd-8199-7f2a95232aa8\") " pod="kube-system/global-pull-secret-syncer-b2gxt" Sep 09 22:59:01.287420 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.287297 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a3574ed6-8639-4a22-b118-ebb5500b086d-etc-sysconfig\") pod \"tuned-cb4k2\" (UID: \"a3574ed6-8639-4a22-b118-ebb5500b086d\") " pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.287420 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.287339 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b43af431-82d9-45b1-937a-5c1cede57ac2-host-slash\") pod \"iptables-alerter-7mz5t\" (UID: \"b43af431-82d9-45b1-937a-5c1cede57ac2\") " pod="openshift-network-operator/iptables-alerter-7mz5t" Sep 09 22:59:01.287420 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.287344 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pd27g\" (UniqueName: \"kubernetes.io/projected/b85f7fcb-c8f2-43aa-9d28-890c87afcdd0-kube-api-access-pd27g\") pod \"network-check-target-gdhb9\" (UID: \"b85f7fcb-c8f2-43aa-9d28-890c87afcdd0\") " pod="openshift-network-diagnostics/network-check-target-gdhb9" Sep 09 22:59:01.287420 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.287367 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/785b3628-74d5-4a5a-b3c5-10b7d2c72140-etc-selinux\") pod \"aws-ebs-csi-driver-node-5pckv\" (UID: \"785b3628-74d5-4a5a-b3c5-10b7d2c72140\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pckv" Sep 09 22:59:01.287420 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.287417 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a3574ed6-8639-4a22-b118-ebb5500b086d-etc-sysconfig\") pod \"tuned-cb4k2\" (UID: \"a3574ed6-8639-4a22-b118-ebb5500b086d\") " pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.288193 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.287299 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/66786687-e3fe-42fd-8199-7f2a95232aa8-dbus\") pod \"global-pull-secret-syncer-b2gxt\" (UID: \"66786687-e3fe-42fd-8199-7f2a95232aa8\") " pod="kube-system/global-pull-secret-syncer-b2gxt" Sep 09 22:59:01.288193 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:01.287448 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 22:59:01.288193 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:01.287497 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66786687-e3fe-42fd-8199-7f2a95232aa8-original-pull-secret podName:66786687-e3fe-42fd-8199-7f2a95232aa8 nodeName:}" failed. No retries permitted until 2026-09-09 22:59:01.787479934 +0000 UTC m=+3.137346949 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/66786687-e3fe-42fd-8199-7f2a95232aa8-original-pull-secret") pod "global-pull-secret-syncer-b2gxt" (UID: "66786687-e3fe-42fd-8199-7f2a95232aa8") : object "kube-system"/"original-pull-secret" not registered Sep 09 22:59:01.288193 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.287626 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/785b3628-74d5-4a5a-b3c5-10b7d2c72140-device-dir\") pod \"aws-ebs-csi-driver-node-5pckv\" (UID: \"785b3628-74d5-4a5a-b3c5-10b7d2c72140\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pckv" Sep 09 22:59:01.288193 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.287667 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/785b3628-74d5-4a5a-b3c5-10b7d2c72140-sys-fs\") pod \"aws-ebs-csi-driver-node-5pckv\" (UID: \"785b3628-74d5-4a5a-b3c5-10b7d2c72140\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pckv" Sep 09 22:59:01.288193 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.287689 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3574ed6-8639-4a22-b118-ebb5500b086d-etc-kubernetes\") pod \"tuned-cb4k2\" (UID: \"a3574ed6-8639-4a22-b118-ebb5500b086d\") " pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.288193 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.287709 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a3574ed6-8639-4a22-b118-ebb5500b086d-run\") pod \"tuned-cb4k2\" (UID: \"a3574ed6-8639-4a22-b118-ebb5500b086d\") " pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.288193 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.287722 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/785b3628-74d5-4a5a-b3c5-10b7d2c72140-device-dir\") pod \"aws-ebs-csi-driver-node-5pckv\" (UID: \"785b3628-74d5-4a5a-b3c5-10b7d2c72140\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pckv" Sep 09 22:59:01.288193 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.287748 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b43af431-82d9-45b1-937a-5c1cede57ac2-iptables-alerter-script\") pod \"iptables-alerter-7mz5t\" (UID: \"b43af431-82d9-45b1-937a-5c1cede57ac2\") " pod="openshift-network-operator/iptables-alerter-7mz5t" Sep 09 22:59:01.288193 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.287761 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/785b3628-74d5-4a5a-b3c5-10b7d2c72140-sys-fs\") pod \"aws-ebs-csi-driver-node-5pckv\" (UID: \"785b3628-74d5-4a5a-b3c5-10b7d2c72140\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pckv" Sep 09 22:59:01.288193 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.287778 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3574ed6-8639-4a22-b118-ebb5500b086d-etc-kubernetes\") pod \"tuned-cb4k2\" (UID: \"a3574ed6-8639-4a22-b118-ebb5500b086d\") " pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.288193 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.287772 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a3574ed6-8639-4a22-b118-ebb5500b086d-tmp\") pod \"tuned-cb4k2\" (UID: \"a3574ed6-8639-4a22-b118-ebb5500b086d\") " pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.288193 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.287811 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a3574ed6-8639-4a22-b118-ebb5500b086d-run\") pod \"tuned-cb4k2\" (UID: \"a3574ed6-8639-4a22-b118-ebb5500b086d\") " pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.288193 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.287905 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a3574ed6-8639-4a22-b118-ebb5500b086d-etc-sysctl-d\") pod \"tuned-cb4k2\" (UID: \"a3574ed6-8639-4a22-b118-ebb5500b086d\") " pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.288193 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.287815 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a3574ed6-8639-4a22-b118-ebb5500b086d-etc-sysctl-d\") pod \"tuned-cb4k2\" (UID: \"a3574ed6-8639-4a22-b118-ebb5500b086d\") " pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.288193 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.287932 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/785b3628-74d5-4a5a-b3c5-10b7d2c72140-registration-dir\") pod \"aws-ebs-csi-driver-node-5pckv\" (UID: \"785b3628-74d5-4a5a-b3c5-10b7d2c72140\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pckv" Sep 09 22:59:01.288193 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.287946 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a3574ed6-8639-4a22-b118-ebb5500b086d-sys\") pod \"tuned-cb4k2\" (UID: \"a3574ed6-8639-4a22-b118-ebb5500b086d\") " pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.288946 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.287987 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpwqw\" (UniqueName: \"kubernetes.io/projected/a3574ed6-8639-4a22-b118-ebb5500b086d-kube-api-access-xpwqw\") pod \"tuned-cb4k2\" (UID: \"a3574ed6-8639-4a22-b118-ebb5500b086d\") " pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.288946 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.288006 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kr95\" (UniqueName: \"kubernetes.io/projected/b43af431-82d9-45b1-937a-5c1cede57ac2-kube-api-access-4kr95\") pod \"iptables-alerter-7mz5t\" (UID: \"b43af431-82d9-45b1-937a-5c1cede57ac2\") " pod="openshift-network-operator/iptables-alerter-7mz5t" Sep 09 22:59:01.288946 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.288070 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a3574ed6-8639-4a22-b118-ebb5500b086d-sys\") pod \"tuned-cb4k2\" (UID: \"a3574ed6-8639-4a22-b118-ebb5500b086d\") " pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.288946 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.288107 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/785b3628-74d5-4a5a-b3c5-10b7d2c72140-registration-dir\") pod \"aws-ebs-csi-driver-node-5pckv\" (UID: \"785b3628-74d5-4a5a-b3c5-10b7d2c72140\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pckv" Sep 09 22:59:01.288946 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.288287 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b43af431-82d9-45b1-937a-5c1cede57ac2-iptables-alerter-script\") pod \"iptables-alerter-7mz5t\" (UID: \"b43af431-82d9-45b1-937a-5c1cede57ac2\") " pod="openshift-network-operator/iptables-alerter-7mz5t" Sep 09 22:59:01.290264 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.290188 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a3574ed6-8639-4a22-b118-ebb5500b086d-etc-tuned\") pod \"tuned-cb4k2\" (UID: \"a3574ed6-8639-4a22-b118-ebb5500b086d\") " pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.290264 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.290224 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a3574ed6-8639-4a22-b118-ebb5500b086d-tmp\") pod \"tuned-cb4k2\" (UID: \"a3574ed6-8639-4a22-b118-ebb5500b086d\") " pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.292890 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:01.292827 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 22:59:01.292890 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:01.292852 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 22:59:01.292890 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:01.292865 2580 projected.go:196] Error preparing data for projected volume kube-api-access-pd27g for pod openshift-network-diagnostics/network-check-target-gdhb9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 22:59:01.293103 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:01.292929 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b85f7fcb-c8f2-43aa-9d28-890c87afcdd0-kube-api-access-pd27g podName:b85f7fcb-c8f2-43aa-9d28-890c87afcdd0 nodeName:}" failed. No retries permitted until 2026-09-09 22:59:01.792912745 +0000 UTC m=+3.142779777 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-pd27g" (UniqueName: "kubernetes.io/projected/b85f7fcb-c8f2-43aa-9d28-890c87afcdd0-kube-api-access-pd27g") pod "network-check-target-gdhb9" (UID: "b85f7fcb-c8f2-43aa-9d28-890c87afcdd0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 22:59:01.295285 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.295227 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cv2ml\" (UniqueName: \"kubernetes.io/projected/785b3628-74d5-4a5a-b3c5-10b7d2c72140-kube-api-access-cv2ml\") pod \"aws-ebs-csi-driver-node-5pckv\" (UID: \"785b3628-74d5-4a5a-b3c5-10b7d2c72140\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pckv" Sep 09 22:59:01.295961 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.295940 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpwqw\" (UniqueName: \"kubernetes.io/projected/a3574ed6-8639-4a22-b118-ebb5500b086d-kube-api-access-xpwqw\") pod \"tuned-cb4k2\" (UID: \"a3574ed6-8639-4a22-b118-ebb5500b086d\") " pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.296062 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.296034 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kr95\" (UniqueName: \"kubernetes.io/projected/b43af431-82d9-45b1-937a-5c1cede57ac2-kube-api-access-4kr95\") pod \"iptables-alerter-7mz5t\" (UID: \"b43af431-82d9-45b1-937a-5c1cede57ac2\") " pod="openshift-network-operator/iptables-alerter-7mz5t" Sep 09 22:59:01.366077 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.366048 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jx58q" Sep 09 22:59:01.375922 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.375897 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ps2b2" Sep 09 22:59:01.387428 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.387396 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dc58c" Sep 09 22:59:01.394131 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.394106 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:01.400560 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.400542 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lwqr2" Sep 09 22:59:01.408054 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.408032 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-w9lv9" Sep 09 22:59:01.414448 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.414429 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pckv" Sep 09 22:59:01.421837 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.421815 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" Sep 09 22:59:01.429236 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.429218 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7mz5t" Sep 09 22:59:01.488215 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.488188 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 22:59:01.690537 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.690502 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cbe663c-0531-4339-b48b-c4d4c3dd5c2b-metrics-certs\") pod \"network-metrics-daemon-8dwkd\" (UID: \"7cbe663c-0531-4339-b48b-c4d4c3dd5c2b\") " pod="openshift-multus/network-metrics-daemon-8dwkd" Sep 09 22:59:01.690708 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:01.690648 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 22:59:01.690754 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:01.690715 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cbe663c-0531-4339-b48b-c4d4c3dd5c2b-metrics-certs podName:7cbe663c-0531-4339-b48b-c4d4c3dd5c2b nodeName:}" failed. No retries permitted until 2026-09-09 22:59:02.69069756 +0000 UTC m=+4.040564592 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7cbe663c-0531-4339-b48b-c4d4c3dd5c2b-metrics-certs") pod "network-metrics-daemon-8dwkd" (UID: "7cbe663c-0531-4339-b48b-c4d4c3dd5c2b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 22:59:01.791519 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.791481 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/66786687-e3fe-42fd-8199-7f2a95232aa8-original-pull-secret\") pod \"global-pull-secret-syncer-b2gxt\" (UID: \"66786687-e3fe-42fd-8199-7f2a95232aa8\") " pod="kube-system/global-pull-secret-syncer-b2gxt" Sep 09 22:59:01.791681 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:01.791590 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 22:59:01.791681 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:01.791647 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66786687-e3fe-42fd-8199-7f2a95232aa8-original-pull-secret podName:66786687-e3fe-42fd-8199-7f2a95232aa8 nodeName:}" failed. No retries permitted until 2026-09-09 22:59:02.791630019 +0000 UTC m=+4.141497035 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/66786687-e3fe-42fd-8199-7f2a95232aa8-original-pull-secret") pod "global-pull-secret-syncer-b2gxt" (UID: "66786687-e3fe-42fd-8199-7f2a95232aa8") : object "kube-system"/"original-pull-secret" not registered Sep 09 22:59:01.892165 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:01.892129 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pd27g\" (UniqueName: \"kubernetes.io/projected/b85f7fcb-c8f2-43aa-9d28-890c87afcdd0-kube-api-access-pd27g\") pod \"network-check-target-gdhb9\" (UID: \"b85f7fcb-c8f2-43aa-9d28-890c87afcdd0\") " pod="openshift-network-diagnostics/network-check-target-gdhb9" Sep 09 22:59:01.892306 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:01.892253 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 22:59:01.892306 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:01.892267 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 22:59:01.892306 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:01.892275 2580 projected.go:196] Error preparing data for projected volume kube-api-access-pd27g for pod openshift-network-diagnostics/network-check-target-gdhb9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 22:59:01.892433 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:01.892334 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b85f7fcb-c8f2-43aa-9d28-890c87afcdd0-kube-api-access-pd27g podName:b85f7fcb-c8f2-43aa-9d28-890c87afcdd0 nodeName:}" failed. No retries permitted until 2026-09-09 22:59:02.892312346 +0000 UTC m=+4.242179358 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-pd27g" (UniqueName: "kubernetes.io/projected/b85f7fcb-c8f2-43aa-9d28-890c87afcdd0-kube-api-access-pd27g") pod "network-check-target-gdhb9" (UID: "b85f7fcb-c8f2-43aa-9d28-890c87afcdd0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 22:59:02.130621 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:02.130579 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 22:54:01 +0000 UTC" deadline="2028-04-14 00:19:44.549664212 +0000 UTC" Sep 09 22:59:02.130621 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:02.130614 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13969h20m42.419053013s" Sep 09 22:59:02.172893 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:02.172861 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gdhb9" Sep 09 22:59:02.173069 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:02.172989 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gdhb9" podUID="b85f7fcb-c8f2-43aa-9d28-890c87afcdd0" Sep 09 22:59:02.247102 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:59:02.246990 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09c46a31_75e9_40a5_856f_4f53ae4b0a3b.slice/crio-bbe7ccb4ef80a5f6f390e802c7e0e1d5998c0f6bf8cf10af861c8c2b9f1b6695 WatchSource:0}: Error finding container bbe7ccb4ef80a5f6f390e802c7e0e1d5998c0f6bf8cf10af861c8c2b9f1b6695: Status 404 returned error can't find the container with id bbe7ccb4ef80a5f6f390e802c7e0e1d5998c0f6bf8cf10af861c8c2b9f1b6695 Sep 09 22:59:02.248818 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:59:02.248790 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddadd077d_e65f_49c0_a1bf_b8299ef2bb89.slice/crio-53e6bc3790e517f1d9b37b58263769637d2bd2fffdf344f9b35c97a317a77033 WatchSource:0}: Error finding container 53e6bc3790e517f1d9b37b58263769637d2bd2fffdf344f9b35c97a317a77033: Status 404 returned error can't find the container with id 53e6bc3790e517f1d9b37b58263769637d2bd2fffdf344f9b35c97a317a77033 Sep 09 22:59:02.249539 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:59:02.249418 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3574ed6_8639_4a22_b118_ebb5500b086d.slice/crio-515f9e080b3c8b3c25bbb1b1bc0a212c7d9ffed4c856c54ff4703647c4635bdd WatchSource:0}: Error finding container 515f9e080b3c8b3c25bbb1b1bc0a212c7d9ffed4c856c54ff4703647c4635bdd: Status 404 returned error can't find the container with id 515f9e080b3c8b3c25bbb1b1bc0a212c7d9ffed4c856c54ff4703647c4635bdd Sep 09 22:59:02.251420 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:59:02.251343 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a2906d3_5dc1_441e_b7f8_a97e2a67d4ed.slice/crio-5b2695bbbec0822b93c0064facf1cb1b33e895a28f8ab1ae7db32f1374eae551 WatchSource:0}: Error finding container 5b2695bbbec0822b93c0064facf1cb1b33e895a28f8ab1ae7db32f1374eae551: Status 404 returned error can't find the container with id 5b2695bbbec0822b93c0064facf1cb1b33e895a28f8ab1ae7db32f1374eae551 Sep 09 22:59:02.252708 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:59:02.252688 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb43af431_82d9_45b1_937a_5c1cede57ac2.slice/crio-b9bac85166581f34287a00424b7cc3cec307b4ce4b1c68afc0d3024d4eee417e WatchSource:0}: Error finding container b9bac85166581f34287a00424b7cc3cec307b4ce4b1c68afc0d3024d4eee417e: Status 404 returned error can't find the container with id b9bac85166581f34287a00424b7cc3cec307b4ce4b1c68afc0d3024d4eee417e Sep 09 22:59:02.254093 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:59:02.254069 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod785b3628_74d5_4a5a_b3c5_10b7d2c72140.slice/crio-cd90890b7ff05001c8efb4bac24b98ec69687ba083892a76e5207467dcc8a5f3 WatchSource:0}: Error finding container cd90890b7ff05001c8efb4bac24b98ec69687ba083892a76e5207467dcc8a5f3: Status 404 returned error can't find the container with id cd90890b7ff05001c8efb4bac24b98ec69687ba083892a76e5207467dcc8a5f3 Sep 09 22:59:02.255408 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:59:02.255100 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbece1799_8601_448b_9fb5_67fe028e3d7e.slice/crio-e932e2faf019089ec50c93eaff8c815b2295b7eab4b8e9de4573d75283c7eae3 WatchSource:0}: Error finding container e932e2faf019089ec50c93eaff8c815b2295b7eab4b8e9de4573d75283c7eae3: Status 404 returned error can't find the container with id e932e2faf019089ec50c93eaff8c815b2295b7eab4b8e9de4573d75283c7eae3 Sep 09 22:59:02.256884 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:59:02.256866 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcc18d79_d518_4d89_ac61_0e3e1ff55794.slice/crio-91dc16d1076db8013a23e52911986c560fe4ae12dd0e5e214357c77a0aebeb17 WatchSource:0}: Error finding container 91dc16d1076db8013a23e52911986c560fe4ae12dd0e5e214357c77a0aebeb17: Status 404 returned error can't find the container with id 91dc16d1076db8013a23e52911986c560fe4ae12dd0e5e214357c77a0aebeb17 Sep 09 22:59:02.258606 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:59:02.258587 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fe9b158_c1b2_498d_aecc_d32aee3c51c6.slice/crio-8b728a59614b6cf80bde290dbc4be663c779f35d9ebdf3bfd01d2ebb6715cc17 WatchSource:0}: Error finding container 8b728a59614b6cf80bde290dbc4be663c779f35d9ebdf3bfd01d2ebb6715cc17: Status 404 returned error can't find the container with id 8b728a59614b6cf80bde290dbc4be663c779f35d9ebdf3bfd01d2ebb6715cc17 Sep 09 22:59:02.696989 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:02.696938 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cbe663c-0531-4339-b48b-c4d4c3dd5c2b-metrics-certs\") pod \"network-metrics-daemon-8dwkd\" (UID: \"7cbe663c-0531-4339-b48b-c4d4c3dd5c2b\") " pod="openshift-multus/network-metrics-daemon-8dwkd" Sep 09 22:59:02.697155 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:02.697054 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 22:59:02.697155 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:02.697103 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cbe663c-0531-4339-b48b-c4d4c3dd5c2b-metrics-certs podName:7cbe663c-0531-4339-b48b-c4d4c3dd5c2b nodeName:}" failed. No retries permitted until 2026-09-09 22:59:04.69708949 +0000 UTC m=+6.046956501 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7cbe663c-0531-4339-b48b-c4d4c3dd5c2b-metrics-certs") pod "network-metrics-daemon-8dwkd" (UID: "7cbe663c-0531-4339-b48b-c4d4c3dd5c2b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 22:59:02.797998 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:02.797887 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/66786687-e3fe-42fd-8199-7f2a95232aa8-original-pull-secret\") pod \"global-pull-secret-syncer-b2gxt\" (UID: \"66786687-e3fe-42fd-8199-7f2a95232aa8\") " pod="kube-system/global-pull-secret-syncer-b2gxt" Sep 09 22:59:02.798154 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:02.798075 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 22:59:02.798154 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:02.798151 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66786687-e3fe-42fd-8199-7f2a95232aa8-original-pull-secret podName:66786687-e3fe-42fd-8199-7f2a95232aa8 nodeName:}" failed. No retries permitted until 2026-09-09 22:59:04.798130964 +0000 UTC m=+6.147997977 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/66786687-e3fe-42fd-8199-7f2a95232aa8-original-pull-secret") pod "global-pull-secret-syncer-b2gxt" (UID: "66786687-e3fe-42fd-8199-7f2a95232aa8") : object "kube-system"/"original-pull-secret" not registered Sep 09 22:59:02.881376 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:02.881342 2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 22:59:02.898680 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:02.898642 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pd27g\" (UniqueName: \"kubernetes.io/projected/b85f7fcb-c8f2-43aa-9d28-890c87afcdd0-kube-api-access-pd27g\") pod \"network-check-target-gdhb9\" (UID: \"b85f7fcb-c8f2-43aa-9d28-890c87afcdd0\") " pod="openshift-network-diagnostics/network-check-target-gdhb9" Sep 09 22:59:02.898853 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:02.898822 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 22:59:02.898853 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:02.898846 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 22:59:02.898987 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:02.898859 2580 projected.go:196] Error preparing data for projected volume kube-api-access-pd27g for pod openshift-network-diagnostics/network-check-target-gdhb9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 22:59:02.898987 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:02.898917 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b85f7fcb-c8f2-43aa-9d28-890c87afcdd0-kube-api-access-pd27g podName:b85f7fcb-c8f2-43aa-9d28-890c87afcdd0 nodeName:}" failed. No retries permitted until 2026-09-09 22:59:04.898899728 +0000 UTC m=+6.248766757 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-pd27g" (UniqueName: "kubernetes.io/projected/b85f7fcb-c8f2-43aa-9d28-890c87afcdd0-kube-api-access-pd27g") pod "network-check-target-gdhb9" (UID: "b85f7fcb-c8f2-43aa-9d28-890c87afcdd0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 22:59:03.131485 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:03.131384 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 22:54:01 +0000 UTC" deadline="2028-03-08 20:35:00.435148948 +0000 UTC" Sep 09 22:59:03.131485 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:03.131448 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13101h35m57.303705384s" Sep 09 22:59:03.174727 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:03.174118 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2gxt" Sep 09 22:59:03.174727 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:03.174151 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dwkd" Sep 09 22:59:03.174727 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:03.174253 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2gxt" podUID="66786687-e3fe-42fd-8199-7f2a95232aa8" Sep 09 22:59:03.174727 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:03.174686 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dwkd" podUID="7cbe663c-0531-4339-b48b-c4d4c3dd5c2b" Sep 09 22:59:03.213269 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:03.213204 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" event={"ID":"bece1799-8601-448b-9fb5-67fe028e3d7e","Type":"ContainerStarted","Data":"e932e2faf019089ec50c93eaff8c815b2295b7eab4b8e9de4573d75283c7eae3"} Sep 09 22:59:03.234370 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:03.234328 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pckv" event={"ID":"785b3628-74d5-4a5a-b3c5-10b7d2c72140","Type":"ContainerStarted","Data":"cd90890b7ff05001c8efb4bac24b98ec69687ba083892a76e5207467dcc8a5f3"} Sep 09 22:59:03.238352 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:03.238314 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7mz5t" event={"ID":"b43af431-82d9-45b1-937a-5c1cede57ac2","Type":"ContainerStarted","Data":"b9bac85166581f34287a00424b7cc3cec307b4ce4b1c68afc0d3024d4eee417e"} Sep 09 22:59:03.247183 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:03.247142 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" event={"ID":"a3574ed6-8639-4a22-b118-ebb5500b086d","Type":"ContainerStarted","Data":"515f9e080b3c8b3c25bbb1b1bc0a212c7d9ffed4c856c54ff4703647c4635bdd"} Sep 09 22:59:03.255112 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:03.255075 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ps2b2" event={"ID":"2a2906d3-5dc1-441e-b7f8-a97e2a67d4ed","Type":"ContainerStarted","Data":"5b2695bbbec0822b93c0064facf1cb1b33e895a28f8ab1ae7db32f1374eae551"} Sep 09 22:59:03.258569 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:03.258538 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jx58q" event={"ID":"0fe9b158-c1b2-498d-aecc-d32aee3c51c6","Type":"ContainerStarted","Data":"8b728a59614b6cf80bde290dbc4be663c779f35d9ebdf3bfd01d2ebb6715cc17"} Sep 09 22:59:03.268348 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:03.268245 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-w9lv9" event={"ID":"dcc18d79-d518-4d89-ac61-0e3e1ff55794","Type":"ContainerStarted","Data":"91dc16d1076db8013a23e52911986c560fe4ae12dd0e5e214357c77a0aebeb17"} Sep 09 22:59:03.278314 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:03.278225 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lwqr2" event={"ID":"dadd077d-e65f-49c0-a1bf-b8299ef2bb89","Type":"ContainerStarted","Data":"53e6bc3790e517f1d9b37b58263769637d2bd2fffdf344f9b35c97a317a77033"} Sep 09 22:59:03.282426 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:03.282390 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dc58c" event={"ID":"09c46a31-75e9-40a5-856f-4f53ae4b0a3b","Type":"ContainerStarted","Data":"bbe7ccb4ef80a5f6f390e802c7e0e1d5998c0f6bf8cf10af861c8c2b9f1b6695"} Sep 09 22:59:03.291244 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:03.291209 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-92.ec2.internal" event={"ID":"8d896ae2507ca9bca5f4fea55fdadcb0","Type":"ContainerStarted","Data":"f3fa2f82b257d343127b51a839113691341392d97be0c467686e1307733ad78d"} Sep 09 22:59:03.301796 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:03.301759 2580 generic.go:356] "Generic (PLEG): container finished" podID="fa1cb3f75de266c7e0e4d8ba90349b01" containerID="40aab60d1bd339026e8102a307b5307010f368520793a4fe2e1c7a36c791f0cf" exitCode=0 Sep 09 22:59:03.302394 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:03.302369 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-92.ec2.internal" event={"ID":"fa1cb3f75de266c7e0e4d8ba90349b01","Type":"ContainerDied","Data":"40aab60d1bd339026e8102a307b5307010f368520793a4fe2e1c7a36c791f0cf"} Sep 09 22:59:03.321027 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:03.320042 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-92.ec2.internal" podStartSLOduration=3.320017132 podStartE2EDuration="3.320017132s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 22:59:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 22:59:03.305560791 +0000 UTC m=+4.655427829" watchObservedRunningTime="2026-09-09 22:59:03.320017132 +0000 UTC m=+4.669884181" Sep 09 22:59:04.172957 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:04.172918 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gdhb9" Sep 09 22:59:04.173423 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:04.173076 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gdhb9" podUID="b85f7fcb-c8f2-43aa-9d28-890c87afcdd0" Sep 09 22:59:04.316374 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:04.316339 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-92.ec2.internal" event={"ID":"fa1cb3f75de266c7e0e4d8ba90349b01","Type":"ContainerStarted","Data":"da98374a05e984f74fc36de2081016158f6b791b265eead8b977c313db7d4c16"} Sep 09 22:59:04.713497 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:04.713449 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cbe663c-0531-4339-b48b-c4d4c3dd5c2b-metrics-certs\") pod \"network-metrics-daemon-8dwkd\" (UID: \"7cbe663c-0531-4339-b48b-c4d4c3dd5c2b\") " pod="openshift-multus/network-metrics-daemon-8dwkd" Sep 09 22:59:04.713680 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:04.713647 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 22:59:04.713739 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:04.713715 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cbe663c-0531-4339-b48b-c4d4c3dd5c2b-metrics-certs podName:7cbe663c-0531-4339-b48b-c4d4c3dd5c2b nodeName:}" failed. No retries permitted until 2026-09-09 22:59:08.713695354 +0000 UTC m=+10.063562391 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7cbe663c-0531-4339-b48b-c4d4c3dd5c2b-metrics-certs") pod "network-metrics-daemon-8dwkd" (UID: "7cbe663c-0531-4339-b48b-c4d4c3dd5c2b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 22:59:04.814138 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:04.814091 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/66786687-e3fe-42fd-8199-7f2a95232aa8-original-pull-secret\") pod \"global-pull-secret-syncer-b2gxt\" (UID: \"66786687-e3fe-42fd-8199-7f2a95232aa8\") " pod="kube-system/global-pull-secret-syncer-b2gxt" Sep 09 22:59:04.814347 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:04.814291 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 22:59:04.814422 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:04.814357 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66786687-e3fe-42fd-8199-7f2a95232aa8-original-pull-secret podName:66786687-e3fe-42fd-8199-7f2a95232aa8 nodeName:}" failed. No retries permitted until 2026-09-09 22:59:08.814339778 +0000 UTC m=+10.164206794 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/66786687-e3fe-42fd-8199-7f2a95232aa8-original-pull-secret") pod "global-pull-secret-syncer-b2gxt" (UID: "66786687-e3fe-42fd-8199-7f2a95232aa8") : object "kube-system"/"original-pull-secret" not registered Sep 09 22:59:04.916027 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:04.915985 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pd27g\" (UniqueName: \"kubernetes.io/projected/b85f7fcb-c8f2-43aa-9d28-890c87afcdd0-kube-api-access-pd27g\") pod \"network-check-target-gdhb9\" (UID: \"b85f7fcb-c8f2-43aa-9d28-890c87afcdd0\") " pod="openshift-network-diagnostics/network-check-target-gdhb9" Sep 09 22:59:04.916744 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:04.916187 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 22:59:04.916744 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:04.916209 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 22:59:04.916744 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:04.916222 2580 projected.go:196] Error preparing data for projected volume kube-api-access-pd27g for pod openshift-network-diagnostics/network-check-target-gdhb9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 22:59:04.916744 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:04.916285 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b85f7fcb-c8f2-43aa-9d28-890c87afcdd0-kube-api-access-pd27g podName:b85f7fcb-c8f2-43aa-9d28-890c87afcdd0 nodeName:}" failed. No retries permitted until 2026-09-09 22:59:08.916266174 +0000 UTC m=+10.266133190 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-pd27g" (UniqueName: "kubernetes.io/projected/b85f7fcb-c8f2-43aa-9d28-890c87afcdd0-kube-api-access-pd27g") pod "network-check-target-gdhb9" (UID: "b85f7fcb-c8f2-43aa-9d28-890c87afcdd0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 22:59:05.175381 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:05.175338 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2gxt" Sep 09 22:59:05.175844 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:05.175466 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2gxt" podUID="66786687-e3fe-42fd-8199-7f2a95232aa8" Sep 09 22:59:05.175922 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:05.175851 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dwkd" Sep 09 22:59:05.175993 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:05.175945 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dwkd" podUID="7cbe663c-0531-4339-b48b-c4d4c3dd5c2b" Sep 09 22:59:06.173863 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:06.173339 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gdhb9" Sep 09 22:59:06.173863 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:06.173460 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gdhb9" podUID="b85f7fcb-c8f2-43aa-9d28-890c87afcdd0" Sep 09 22:59:07.173893 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:07.173821 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2gxt" Sep 09 22:59:07.174367 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:07.173959 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2gxt" podUID="66786687-e3fe-42fd-8199-7f2a95232aa8" Sep 09 22:59:07.174367 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:07.174025 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dwkd" Sep 09 22:59:07.174367 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:07.174142 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dwkd" podUID="7cbe663c-0531-4339-b48b-c4d4c3dd5c2b" Sep 09 22:59:08.173221 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:08.173187 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gdhb9" Sep 09 22:59:08.173402 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:08.173324 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gdhb9" podUID="b85f7fcb-c8f2-43aa-9d28-890c87afcdd0" Sep 09 22:59:08.747179 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:08.747087 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cbe663c-0531-4339-b48b-c4d4c3dd5c2b-metrics-certs\") pod \"network-metrics-daemon-8dwkd\" (UID: \"7cbe663c-0531-4339-b48b-c4d4c3dd5c2b\") " pod="openshift-multus/network-metrics-daemon-8dwkd" Sep 09 22:59:08.747758 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:08.747266 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 22:59:08.747758 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:08.747341 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cbe663c-0531-4339-b48b-c4d4c3dd5c2b-metrics-certs podName:7cbe663c-0531-4339-b48b-c4d4c3dd5c2b nodeName:}" failed. No retries permitted until 2026-09-09 22:59:16.74732019 +0000 UTC m=+18.097187217 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7cbe663c-0531-4339-b48b-c4d4c3dd5c2b-metrics-certs") pod "network-metrics-daemon-8dwkd" (UID: "7cbe663c-0531-4339-b48b-c4d4c3dd5c2b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 22:59:08.848319 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:08.848091 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/66786687-e3fe-42fd-8199-7f2a95232aa8-original-pull-secret\") pod \"global-pull-secret-syncer-b2gxt\" (UID: \"66786687-e3fe-42fd-8199-7f2a95232aa8\") " pod="kube-system/global-pull-secret-syncer-b2gxt" Sep 09 22:59:08.848319 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:08.848252 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 22:59:08.848319 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:08.848318 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66786687-e3fe-42fd-8199-7f2a95232aa8-original-pull-secret podName:66786687-e3fe-42fd-8199-7f2a95232aa8 nodeName:}" failed. No retries permitted until 2026-09-09 22:59:16.848299341 +0000 UTC m=+18.198166359 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/66786687-e3fe-42fd-8199-7f2a95232aa8-original-pull-secret") pod "global-pull-secret-syncer-b2gxt" (UID: "66786687-e3fe-42fd-8199-7f2a95232aa8") : object "kube-system"/"original-pull-secret" not registered Sep 09 22:59:08.949252 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:08.948619 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pd27g\" (UniqueName: \"kubernetes.io/projected/b85f7fcb-c8f2-43aa-9d28-890c87afcdd0-kube-api-access-pd27g\") pod \"network-check-target-gdhb9\" (UID: \"b85f7fcb-c8f2-43aa-9d28-890c87afcdd0\") " pod="openshift-network-diagnostics/network-check-target-gdhb9" Sep 09 22:59:08.949252 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:08.948835 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 22:59:08.949252 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:08.948857 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 22:59:08.949252 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:08.948870 2580 projected.go:196] Error preparing data for projected volume kube-api-access-pd27g for pod openshift-network-diagnostics/network-check-target-gdhb9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 22:59:08.949252 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:08.948929 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b85f7fcb-c8f2-43aa-9d28-890c87afcdd0-kube-api-access-pd27g podName:b85f7fcb-c8f2-43aa-9d28-890c87afcdd0 nodeName:}" failed. No retries permitted until 2026-09-09 22:59:16.948910407 +0000 UTC m=+18.298777438 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-pd27g" (UniqueName: "kubernetes.io/projected/b85f7fcb-c8f2-43aa-9d28-890c87afcdd0-kube-api-access-pd27g") pod "network-check-target-gdhb9" (UID: "b85f7fcb-c8f2-43aa-9d28-890c87afcdd0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 22:59:09.174048 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:09.173963 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2gxt" Sep 09 22:59:09.174204 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:09.174070 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dwkd" Sep 09 22:59:09.174204 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:09.174146 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2gxt" podUID="66786687-e3fe-42fd-8199-7f2a95232aa8" Sep 09 22:59:09.174346 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:09.174308 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dwkd" podUID="7cbe663c-0531-4339-b48b-c4d4c3dd5c2b" Sep 09 22:59:10.173702 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:10.173654 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gdhb9" Sep 09 22:59:10.174273 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:10.173800 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gdhb9" podUID="b85f7fcb-c8f2-43aa-9d28-890c87afcdd0" Sep 09 22:59:11.174066 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:11.174026 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2gxt" Sep 09 22:59:11.174496 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:11.174079 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dwkd" Sep 09 22:59:11.174496 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:11.174169 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2gxt" podUID="66786687-e3fe-42fd-8199-7f2a95232aa8" Sep 09 22:59:11.174496 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:11.174307 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dwkd" podUID="7cbe663c-0531-4339-b48b-c4d4c3dd5c2b" Sep 09 22:59:12.174172 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:12.173778 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gdhb9" Sep 09 22:59:12.174172 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:12.173917 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gdhb9" podUID="b85f7fcb-c8f2-43aa-9d28-890c87afcdd0" Sep 09 22:59:13.173095 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:13.173058 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2gxt" Sep 09 22:59:13.173291 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:13.173071 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dwkd" Sep 09 22:59:13.173291 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:13.173194 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2gxt" podUID="66786687-e3fe-42fd-8199-7f2a95232aa8" Sep 09 22:59:13.173382 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:13.173286 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dwkd" podUID="7cbe663c-0531-4339-b48b-c4d4c3dd5c2b" Sep 09 22:59:14.173376 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:14.173333 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gdhb9" Sep 09 22:59:14.173795 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:14.173467 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gdhb9" podUID="b85f7fcb-c8f2-43aa-9d28-890c87afcdd0" Sep 09 22:59:15.173900 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:15.173860 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2gxt" Sep 09 22:59:15.174323 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:15.174024 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2gxt" podUID="66786687-e3fe-42fd-8199-7f2a95232aa8" Sep 09 22:59:15.174323 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:15.174058 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dwkd" Sep 09 22:59:15.174323 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:15.174193 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dwkd" podUID="7cbe663c-0531-4339-b48b-c4d4c3dd5c2b" Sep 09 22:59:16.172864 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:16.172825 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gdhb9" Sep 09 22:59:16.173087 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:16.172945 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gdhb9" podUID="b85f7fcb-c8f2-43aa-9d28-890c87afcdd0" Sep 09 22:59:16.810463 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:16.810420 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cbe663c-0531-4339-b48b-c4d4c3dd5c2b-metrics-certs\") pod \"network-metrics-daemon-8dwkd\" (UID: \"7cbe663c-0531-4339-b48b-c4d4c3dd5c2b\") " pod="openshift-multus/network-metrics-daemon-8dwkd" Sep 09 22:59:16.810942 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:16.810582 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 22:59:16.810942 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:16.810669 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cbe663c-0531-4339-b48b-c4d4c3dd5c2b-metrics-certs podName:7cbe663c-0531-4339-b48b-c4d4c3dd5c2b nodeName:}" failed. No retries permitted until 2026-09-09 22:59:32.810648301 +0000 UTC m=+34.160515313 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7cbe663c-0531-4339-b48b-c4d4c3dd5c2b-metrics-certs") pod "network-metrics-daemon-8dwkd" (UID: "7cbe663c-0531-4339-b48b-c4d4c3dd5c2b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 22:59:16.911740 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:16.911703 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/66786687-e3fe-42fd-8199-7f2a95232aa8-original-pull-secret\") pod \"global-pull-secret-syncer-b2gxt\" (UID: \"66786687-e3fe-42fd-8199-7f2a95232aa8\") " pod="kube-system/global-pull-secret-syncer-b2gxt" Sep 09 22:59:16.911922 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:16.911779 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 22:59:16.911922 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:16.911853 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66786687-e3fe-42fd-8199-7f2a95232aa8-original-pull-secret podName:66786687-e3fe-42fd-8199-7f2a95232aa8 nodeName:}" failed. No retries permitted until 2026-09-09 22:59:32.911832519 +0000 UTC m=+34.261699534 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/66786687-e3fe-42fd-8199-7f2a95232aa8-original-pull-secret") pod "global-pull-secret-syncer-b2gxt" (UID: "66786687-e3fe-42fd-8199-7f2a95232aa8") : object "kube-system"/"original-pull-secret" not registered Sep 09 22:59:17.012295 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:17.012257 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pd27g\" (UniqueName: \"kubernetes.io/projected/b85f7fcb-c8f2-43aa-9d28-890c87afcdd0-kube-api-access-pd27g\") pod \"network-check-target-gdhb9\" (UID: \"b85f7fcb-c8f2-43aa-9d28-890c87afcdd0\") " pod="openshift-network-diagnostics/network-check-target-gdhb9" Sep 09 22:59:17.012458 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:17.012431 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 22:59:17.012458 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:17.012451 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 22:59:17.012527 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:17.012463 2580 projected.go:196] Error preparing data for projected volume kube-api-access-pd27g for pod openshift-network-diagnostics/network-check-target-gdhb9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 22:59:17.012560 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:17.012528 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b85f7fcb-c8f2-43aa-9d28-890c87afcdd0-kube-api-access-pd27g podName:b85f7fcb-c8f2-43aa-9d28-890c87afcdd0 nodeName:}" failed. No retries permitted until 2026-09-09 22:59:33.012509423 +0000 UTC m=+34.362376457 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-pd27g" (UniqueName: "kubernetes.io/projected/b85f7fcb-c8f2-43aa-9d28-890c87afcdd0-kube-api-access-pd27g") pod "network-check-target-gdhb9" (UID: "b85f7fcb-c8f2-43aa-9d28-890c87afcdd0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 22:59:17.173811 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:17.173772 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2gxt" Sep 09 22:59:17.173990 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:17.173777 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dwkd" Sep 09 22:59:17.173990 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:17.173916 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2gxt" podUID="66786687-e3fe-42fd-8199-7f2a95232aa8" Sep 09 22:59:17.174110 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:17.174007 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dwkd" podUID="7cbe663c-0531-4339-b48b-c4d4c3dd5c2b" Sep 09 22:59:18.172925 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:18.172888 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gdhb9" Sep 09 22:59:18.173355 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:18.173019 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gdhb9" podUID="b85f7fcb-c8f2-43aa-9d28-890c87afcdd0" Sep 09 22:59:19.177919 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:19.177880 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dwkd" Sep 09 22:59:19.178405 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:19.177919 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2gxt" Sep 09 22:59:19.178405 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:19.178041 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dwkd" podUID="7cbe663c-0531-4339-b48b-c4d4c3dd5c2b" Sep 09 22:59:19.178405 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:19.178149 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2gxt" podUID="66786687-e3fe-42fd-8199-7f2a95232aa8" Sep 09 22:59:20.174092 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:20.173911 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gdhb9" Sep 09 22:59:20.174235 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:20.174181 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gdhb9" podUID="b85f7fcb-c8f2-43aa-9d28-890c87afcdd0" Sep 09 22:59:20.351255 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:20.351216 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" event={"ID":"bece1799-8601-448b-9fb5-67fe028e3d7e","Type":"ContainerStarted","Data":"743fa965d4e31415a7c1f7f7e35df3584f5941d1fd96dfff6fe1bd755eef8903"} Sep 09 22:59:20.351934 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:20.351291 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" event={"ID":"bece1799-8601-448b-9fb5-67fe028e3d7e","Type":"ContainerStarted","Data":"13b7befd1701f7fdc64be2ec5546256141ea052f36dd1a7089d1e1769962ae78"} Sep 09 22:59:20.351934 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:20.351335 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" event={"ID":"bece1799-8601-448b-9fb5-67fe028e3d7e","Type":"ContainerStarted","Data":"d456d087b061051564ded5721cc1065a1a5d09e32bf9000488d43b53c425ff39"} Sep 09 22:59:20.351934 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:20.351366 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" event={"ID":"bece1799-8601-448b-9fb5-67fe028e3d7e","Type":"ContainerStarted","Data":"e50dbe9bd55860b21a454f94a61d1d703a4bf1550b3cb11bc7a476f27c4a1de3"} Sep 09 22:59:20.351934 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:20.351400 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" event={"ID":"bece1799-8601-448b-9fb5-67fe028e3d7e","Type":"ContainerStarted","Data":"56e3979a3b9e6fd86dff806aee335baaeb8823fc3ebad327ccabba08acf2492f"} Sep 09 22:59:20.352544 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:20.352522 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pckv" event={"ID":"785b3628-74d5-4a5a-b3c5-10b7d2c72140","Type":"ContainerStarted","Data":"f6a16338bb03132ebaa92f1adc25b27de4b371396de576565e4dc093e9e9078e"} Sep 09 22:59:20.353825 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:20.353802 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" event={"ID":"a3574ed6-8639-4a22-b118-ebb5500b086d","Type":"ContainerStarted","Data":"05feee4f6c10ef6ce1333712cc754f709401b376888ce0eb0a791b050db70364"} Sep 09 22:59:20.355379 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:20.355361 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ps2b2" event={"ID":"2a2906d3-5dc1-441e-b7f8-a97e2a67d4ed","Type":"ContainerStarted","Data":"013ea380baccaf453a411e22df1cb207f729a1a7efe80dd41455e436e2c9bdc6"} Sep 09 22:59:20.356830 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:20.356807 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jx58q" event={"ID":"0fe9b158-c1b2-498d-aecc-d32aee3c51c6","Type":"ContainerStarted","Data":"93f87227a0a87b375a6aa367a6678b7a1dfb0aa91bbcf8a1b666a1fdba6817af"} Sep 09 22:59:20.358341 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:20.358318 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-w9lv9" event={"ID":"dcc18d79-d518-4d89-ac61-0e3e1ff55794","Type":"ContainerStarted","Data":"2da3e6c2de183c37cd89dcdadbc846dd1642276dbbd6ac0fea2f732e091f85c3"} Sep 09 22:59:20.359846 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:20.359816 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lwqr2" event={"ID":"dadd077d-e65f-49c0-a1bf-b8299ef2bb89","Type":"ContainerStarted","Data":"4d4a3e326052e7b56bfa169e680cb4125091ef23ef1e91377f120a2cebaadb13"} Sep 09 22:59:20.361328 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:20.361307 2580 generic.go:356] "Generic (PLEG): container finished" podID="09c46a31-75e9-40a5-856f-4f53ae4b0a3b" containerID="3daa0af57835557b62995eea5046594fe565dc6da4113a321dec7713abab521f" exitCode=0 Sep 09 22:59:20.361407 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:20.361341 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dc58c" event={"ID":"09c46a31-75e9-40a5-856f-4f53ae4b0a3b","Type":"ContainerDied","Data":"3daa0af57835557b62995eea5046594fe565dc6da4113a321dec7713abab521f"} Sep 09 22:59:20.369786 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:20.369743 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-92.ec2.internal" podStartSLOduration=20.369730176 podStartE2EDuration="20.369730176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 22:59:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 22:59:04.33118164 +0000 UTC m=+5.681048702" watchObservedRunningTime="2026-09-09 22:59:20.369730176 +0000 UTC m=+21.719597210" Sep 09 22:59:20.384741 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:20.383435 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-jx58q" podStartSLOduration=3.831946619 podStartE2EDuration="21.383414825s" totalImagesPullingTime="17.551468206s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 22:58:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 22:59:20.383235297 +0000 UTC m=+21.733102333" watchObservedRunningTime="2026-09-09 22:59:20.383414825 +0000 UTC m=+21.733281860" Sep 09 22:59:20.384741 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:20.384220 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-cb4k2" podStartSLOduration=3.815545272 podStartE2EDuration="21.384206869s" totalImagesPullingTime="17.568661597s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 22:58:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 22:59:20.369723172 +0000 UTC m=+21.719590205" watchObservedRunningTime="2026-09-09 22:59:20.384206869 +0000 UTC m=+21.734073904" Sep 09 22:59:20.472694 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:20.472501 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ps2b2" podStartSLOduration=3.909623206 podStartE2EDuration="21.472485118s" totalImagesPullingTime="17.562861912s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 22:58:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 22:59:20.455657454 +0000 UTC m=+21.805524489" watchObservedRunningTime="2026-09-09 22:59:20.472485118 +0000 UTC m=+21.822352152" Sep 09 22:59:20.473060 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:20.473036 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-lwqr2" podStartSLOduration=3.9046152640000003 podStartE2EDuration="21.473028492s" totalImagesPullingTime="17.568413228s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 22:58:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 22:59:20.472353752 +0000 UTC m=+21.822220786" watchObservedRunningTime="2026-09-09 22:59:20.473028492 +0000 UTC m=+21.822895538" Sep 09 22:59:20.489669 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:20.489611 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-w9lv9" podStartSLOduration=3.851983916 podStartE2EDuration="21.489597207s" totalImagesPullingTime="17.637613291s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 22:58:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 22:59:20.48904954 +0000 UTC m=+21.838916574" watchObservedRunningTime="2026-09-09 22:59:20.489597207 +0000 UTC m=+21.839464240" Sep 09 22:59:21.173482 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:21.173454 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dwkd" Sep 09 22:59:21.173630 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:21.173583 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dwkd" podUID="7cbe663c-0531-4339-b48b-c4d4c3dd5c2b" Sep 09 22:59:21.173630 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:21.173455 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2gxt" Sep 09 22:59:21.173738 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:21.173658 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2gxt" podUID="66786687-e3fe-42fd-8199-7f2a95232aa8" Sep 09 22:59:21.191552 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:21.191530 2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 09 22:59:21.215169 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:21.215071 2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-09T22:59:21.191547912Z","UUID":"1751f0b2-2589-451c-9c6a-fcf000d40b96","Handler":null,"Name":"","Endpoint":""} Sep 09 22:59:21.216793 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:21.216768 2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 09 22:59:21.216886 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:21.216806 2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 09 22:59:21.366707 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:21.366661 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pckv" event={"ID":"785b3628-74d5-4a5a-b3c5-10b7d2c72140","Type":"ContainerStarted","Data":"8d2f857e38939f41833dbdb590f5b1e57b0f9a13786e4ae328b54aa9eff00286"} Sep 09 22:59:21.368470 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:21.368440 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7mz5t" event={"ID":"b43af431-82d9-45b1-937a-5c1cede57ac2","Type":"ContainerStarted","Data":"54d9b566f3958cf9feb196aa31f49ff78f0f8509a787b33e8379e4b07445c39e"} Sep 09 22:59:21.372127 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:21.372095 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" event={"ID":"bece1799-8601-448b-9fb5-67fe028e3d7e","Type":"ContainerStarted","Data":"0ddccf252b9311f509786a33a120c5366727eef802ced90747e73dd9d0f92a50"} Sep 09 22:59:21.382800 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:21.382750 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-7mz5t" podStartSLOduration=4.818081342 podStartE2EDuration="22.382731695s" totalImagesPullingTime="17.564650353s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 22:58:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 22:59:21.382650839 +0000 UTC m=+22.732517874" watchObservedRunningTime="2026-09-09 22:59:21.382731695 +0000 UTC m=+22.732598730" Sep 09 22:59:22.172960 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:22.172929 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gdhb9" Sep 09 22:59:22.173154 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:22.173041 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gdhb9" podUID="b85f7fcb-c8f2-43aa-9d28-890c87afcdd0" Sep 09 22:59:22.906054 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:22.906014 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ps2b2" Sep 09 22:59:22.906751 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:22.906726 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ps2b2" Sep 09 22:59:23.174039 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:23.173951 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dwkd" Sep 09 22:59:23.174204 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:23.173952 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2gxt" Sep 09 22:59:23.174204 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:23.174111 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dwkd" podUID="7cbe663c-0531-4339-b48b-c4d4c3dd5c2b" Sep 09 22:59:23.174295 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:23.174199 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2gxt" podUID="66786687-e3fe-42fd-8199-7f2a95232aa8" Sep 09 22:59:23.380987 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:23.380935 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" event={"ID":"bece1799-8601-448b-9fb5-67fe028e3d7e","Type":"ContainerStarted","Data":"bd8ddea5119638038572d5339444ea431a2e21a3465d5695cc690e545b11ad19"} Sep 09 22:59:23.383479 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:23.383437 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pckv" event={"ID":"785b3628-74d5-4a5a-b3c5-10b7d2c72140","Type":"ContainerStarted","Data":"e5ab1f548ea615a12c9bf4f8b0003506ec38e06de41ad59ea8ed21a4111a8cf3"} Sep 09 22:59:23.383629 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:23.383610 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ps2b2" Sep 09 22:59:23.384261 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:23.384243 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ps2b2" Sep 09 22:59:23.401986 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:23.401938 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pckv" podStartSLOduration=4.63063769 podStartE2EDuration="24.401926267s" totalImagesPullingTime="19.771288577s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 22:58:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 22:59:23.401866984 +0000 UTC m=+24.751734019" watchObservedRunningTime="2026-09-09 22:59:23.401926267 +0000 UTC m=+24.751793301" Sep 09 22:59:24.173117 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:24.173085 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gdhb9" Sep 09 22:59:24.173513 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:24.173194 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gdhb9" podUID="b85f7fcb-c8f2-43aa-9d28-890c87afcdd0" Sep 09 22:59:25.173298 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:25.173166 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2gxt" Sep 09 22:59:25.173653 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:25.173168 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dwkd" Sep 09 22:59:25.173653 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:25.173377 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2gxt" podUID="66786687-e3fe-42fd-8199-7f2a95232aa8" Sep 09 22:59:25.173653 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:25.173451 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dwkd" podUID="7cbe663c-0531-4339-b48b-c4d4c3dd5c2b" Sep 09 22:59:25.392991 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:25.392929 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" event={"ID":"bece1799-8601-448b-9fb5-67fe028e3d7e","Type":"ContainerStarted","Data":"6c5ed638d79bd190ad82a88b8429bcd57de658f62e890d852347c725768bc1b0"} Sep 09 22:59:25.421549 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:25.421504 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" podStartSLOduration=8.782488877 podStartE2EDuration="26.42148706s" totalImagesPullingTime="17.638998183s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 22:58:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 22:59:25.421029554 +0000 UTC m=+26.770896589" watchObservedRunningTime="2026-09-09 22:59:25.42148706 +0000 UTC m=+26.771354095" Sep 09 22:59:26.173098 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:26.173065 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gdhb9" Sep 09 22:59:26.173236 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:26.173166 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gdhb9" podUID="b85f7fcb-c8f2-43aa-9d28-890c87afcdd0" Sep 09 22:59:26.397296 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:26.397263 2580 generic.go:356] "Generic (PLEG): container finished" podID="09c46a31-75e9-40a5-856f-4f53ae4b0a3b" containerID="e6bae9dce069272f459368ffb903af8a5636b40606d0cf4f4c1dd49d039ab4e6" exitCode=0 Sep 09 22:59:26.397868 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:26.397346 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dc58c" event={"ID":"09c46a31-75e9-40a5-856f-4f53ae4b0a3b","Type":"ContainerDied","Data":"e6bae9dce069272f459368ffb903af8a5636b40606d0cf4f4c1dd49d039ab4e6"} Sep 09 22:59:26.397868 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:26.397856 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:26.398039 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:26.397922 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:26.398039 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:26.397983 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:26.413877 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:26.413855 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:26.414176 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:26.414153 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 22:59:27.100166 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:27.099731 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gdhb9"] Sep 09 22:59:27.100166 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:27.100110 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gdhb9" Sep 09 22:59:27.100479 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:27.100221 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gdhb9" podUID="b85f7fcb-c8f2-43aa-9d28-890c87afcdd0" Sep 09 22:59:27.103926 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:27.103881 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-b2gxt"] Sep 09 22:59:27.104101 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:27.104060 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2gxt" Sep 09 22:59:27.104181 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:27.104155 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2gxt" podUID="66786687-e3fe-42fd-8199-7f2a95232aa8" Sep 09 22:59:27.104644 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:27.104620 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8dwkd"] Sep 09 22:59:27.104752 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:27.104738 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dwkd" Sep 09 22:59:27.104898 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:27.104837 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dwkd" podUID="7cbe663c-0531-4339-b48b-c4d4c3dd5c2b" Sep 09 22:59:28.404495 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:28.404419 2580 generic.go:356] "Generic (PLEG): container finished" podID="09c46a31-75e9-40a5-856f-4f53ae4b0a3b" containerID="08c32648a4a3dd32825c0303d2bc3b66aca2b7877afec589a57497701f8cb23a" exitCode=0 Sep 09 22:59:28.404950 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:28.404498 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dc58c" event={"ID":"09c46a31-75e9-40a5-856f-4f53ae4b0a3b","Type":"ContainerDied","Data":"08c32648a4a3dd32825c0303d2bc3b66aca2b7877afec589a57497701f8cb23a"} Sep 09 22:59:29.174753 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:29.174719 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2gxt" Sep 09 22:59:29.174921 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:29.174762 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dwkd" Sep 09 22:59:29.174921 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:29.174813 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2gxt" podUID="66786687-e3fe-42fd-8199-7f2a95232aa8" Sep 09 22:59:29.174921 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:29.174877 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dwkd" podUID="7cbe663c-0531-4339-b48b-c4d4c3dd5c2b" Sep 09 22:59:29.174921 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:29.174893 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gdhb9" Sep 09 22:59:29.175085 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:29.175005 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gdhb9" podUID="b85f7fcb-c8f2-43aa-9d28-890c87afcdd0" Sep 09 22:59:29.409275 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:29.409042 2580 generic.go:356] "Generic (PLEG): container finished" podID="09c46a31-75e9-40a5-856f-4f53ae4b0a3b" containerID="13a03143c88bad65a53a82875a0ac4e22dd988fe2239359ce483b7bb4fbde61a" exitCode=0 Sep 09 22:59:29.409275 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:29.409123 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dc58c" event={"ID":"09c46a31-75e9-40a5-856f-4f53ae4b0a3b","Type":"ContainerDied","Data":"13a03143c88bad65a53a82875a0ac4e22dd988fe2239359ce483b7bb4fbde61a"} Sep 09 22:59:31.173399 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:31.173368 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dwkd" Sep 09 22:59:31.173767 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:31.173368 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2gxt" Sep 09 22:59:31.173767 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:31.173484 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dwkd" podUID="7cbe663c-0531-4339-b48b-c4d4c3dd5c2b" Sep 09 22:59:31.173767 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:31.173577 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2gxt" podUID="66786687-e3fe-42fd-8199-7f2a95232aa8" Sep 09 22:59:31.173767 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:31.173375 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gdhb9" Sep 09 22:59:31.173767 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:31.173670 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gdhb9" podUID="b85f7fcb-c8f2-43aa-9d28-890c87afcdd0" Sep 09 22:59:32.428236 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.428199 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-92.ec2.internal" event="NodeReady" Sep 09 22:59:32.428631 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.428332 2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 09 22:59:32.460210 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.460181 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-69855d874b-xvwj9"] Sep 09 22:59:32.480808 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.480763 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-zmvn9"] Sep 09 22:59:32.481290 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.481120 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69855d874b-xvwj9" Sep 09 22:59:32.485637 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.485609 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 09 22:59:32.486084 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.486052 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 09 22:59:32.486194 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.486175 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-p5vxk\"" Sep 09 22:59:32.486260 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.486175 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 09 22:59:32.491679 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.491659 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 09 22:59:32.492092 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.492073 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-zmvn9"] Sep 09 22:59:32.492181 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.492108 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-69855d874b-xvwj9"] Sep 09 22:59:32.492181 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.492130 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-b6zvm"] Sep 09 22:59:32.492286 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.492197 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-zmvn9" Sep 09 22:59:32.494535 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.494512 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Sep 09 22:59:32.495001 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.494778 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-wjdkt\"" Sep 09 22:59:32.495001 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.494778 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Sep 09 22:59:32.505608 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.505585 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5dl4c"] Sep 09 22:59:32.505737 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.505702 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b6zvm" Sep 09 22:59:32.508147 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.508112 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-t7bbt\"" Sep 09 22:59:32.508262 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.508221 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 09 22:59:32.508329 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.508312 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 09 22:59:32.519608 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.519587 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5dl4c"] Sep 09 22:59:32.519706 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.519622 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-b6zvm"] Sep 09 22:59:32.519706 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.519684 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5dl4c" Sep 09 22:59:32.522074 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.522046 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 09 22:59:32.522074 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.522066 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 09 22:59:32.522210 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.522052 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-dx997\"" Sep 09 22:59:32.522210 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.522153 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 09 22:59:32.617308 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.617212 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ps2b2_2a2906d3-5dc1-441e-b7f8-a97e2a67d4ed/konnectivity-agent/0.log" limit=1024 Sep 09 22:59:32.621085 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.621061 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6251fed2-5675-42f5-b0d1-788e1a894092-metrics-tls\") pod \"dns-default-b6zvm\" (UID: \"6251fed2-5675-42f5-b0d1-788e1a894092\") " pod="openshift-dns/dns-default-b6zvm" Sep 09 22:59:32.621208 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.621095 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-registry-tls\") pod \"image-registry-69855d874b-xvwj9\" (UID: \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\") " pod="openshift-image-registry/image-registry-69855d874b-xvwj9" Sep 09 22:59:32.621208 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.621120 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-bound-sa-token\") pod \"image-registry-69855d874b-xvwj9\" (UID: \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\") " pod="openshift-image-registry/image-registry-69855d874b-xvwj9" Sep 09 22:59:32.621208 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.621181 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5f001595-27ff-43d7-94d2-7ddc18d99671-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-zmvn9\" (UID: \"5f001595-27ff-43d7-94d2-7ddc18d99671\") " pod="openshift-network-console/networking-console-plugin-79977f9844-zmvn9" Sep 09 22:59:32.621355 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.621234 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-289cb\" (UniqueName: \"kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-kube-api-access-289cb\") pod \"image-registry-69855d874b-xvwj9\" (UID: \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\") " pod="openshift-image-registry/image-registry-69855d874b-xvwj9" Sep 09 22:59:32.621355 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.621264 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1a033d1-6eb8-472d-95dc-5a1c26b60867-cert\") pod \"ingress-canary-5dl4c\" (UID: \"d1a033d1-6eb8-472d-95dc-5a1c26b60867\") " pod="openshift-ingress-canary/ingress-canary-5dl4c" Sep 09 22:59:32.621355 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.621301 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5f001595-27ff-43d7-94d2-7ddc18d99671-nginx-conf\") pod \"networking-console-plugin-79977f9844-zmvn9\" (UID: \"5f001595-27ff-43d7-94d2-7ddc18d99671\") " pod="openshift-network-console/networking-console-plugin-79977f9844-zmvn9" Sep 09 22:59:32.621355 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.621323 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-ca-trust-extracted\") pod \"image-registry-69855d874b-xvwj9\" (UID: \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\") " pod="openshift-image-registry/image-registry-69855d874b-xvwj9" Sep 09 22:59:32.621534 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.621360 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-registry-certificates\") pod \"image-registry-69855d874b-xvwj9\" (UID: \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\") " pod="openshift-image-registry/image-registry-69855d874b-xvwj9" Sep 09 22:59:32.621534 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.621388 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-trusted-ca\") pod \"image-registry-69855d874b-xvwj9\" (UID: \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\") " pod="openshift-image-registry/image-registry-69855d874b-xvwj9" Sep 09 22:59:32.621534 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.621408 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fz65j\" (UniqueName: \"kubernetes.io/projected/d1a033d1-6eb8-472d-95dc-5a1c26b60867-kube-api-access-fz65j\") pod \"ingress-canary-5dl4c\" (UID: \"d1a033d1-6eb8-472d-95dc-5a1c26b60867\") " pod="openshift-ingress-canary/ingress-canary-5dl4c" Sep 09 22:59:32.621534 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.621424 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6251fed2-5675-42f5-b0d1-788e1a894092-tmp-dir\") pod \"dns-default-b6zvm\" (UID: \"6251fed2-5675-42f5-b0d1-788e1a894092\") " pod="openshift-dns/dns-default-b6zvm" Sep 09 22:59:32.621534 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.621447 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-image-registry-private-configuration\") pod \"image-registry-69855d874b-xvwj9\" (UID: \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\") " pod="openshift-image-registry/image-registry-69855d874b-xvwj9" Sep 09 22:59:32.621534 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.621490 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6251fed2-5675-42f5-b0d1-788e1a894092-config-volume\") pod \"dns-default-b6zvm\" (UID: \"6251fed2-5675-42f5-b0d1-788e1a894092\") " pod="openshift-dns/dns-default-b6zvm" Sep 09 22:59:32.621534 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.621526 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-installation-pull-secrets\") pod \"image-registry-69855d874b-xvwj9\" (UID: \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\") " pod="openshift-image-registry/image-registry-69855d874b-xvwj9" Sep 09 22:59:32.621831 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.621574 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8v2w\" (UniqueName: \"kubernetes.io/projected/6251fed2-5675-42f5-b0d1-788e1a894092-kube-api-access-z8v2w\") pod \"dns-default-b6zvm\" (UID: \"6251fed2-5675-42f5-b0d1-788e1a894092\") " pod="openshift-dns/dns-default-b6zvm" Sep 09 22:59:32.695397 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.695361 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ps2b2_2a2906d3-5dc1-441e-b7f8-a97e2a67d4ed/konnectivity-agent/0.log" limit=1024 Sep 09 22:59:32.722466 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.722421 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-registry-certificates\") pod \"image-registry-69855d874b-xvwj9\" (UID: \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\") " pod="openshift-image-registry/image-registry-69855d874b-xvwj9" Sep 09 22:59:32.722466 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.722472 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-trusted-ca\") pod \"image-registry-69855d874b-xvwj9\" (UID: \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\") " pod="openshift-image-registry/image-registry-69855d874b-xvwj9" Sep 09 22:59:32.722682 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.722586 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fz65j\" (UniqueName: \"kubernetes.io/projected/d1a033d1-6eb8-472d-95dc-5a1c26b60867-kube-api-access-fz65j\") pod \"ingress-canary-5dl4c\" (UID: \"d1a033d1-6eb8-472d-95dc-5a1c26b60867\") " pod="openshift-ingress-canary/ingress-canary-5dl4c" Sep 09 22:59:32.722682 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.722633 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6251fed2-5675-42f5-b0d1-788e1a894092-tmp-dir\") pod \"dns-default-b6zvm\" (UID: \"6251fed2-5675-42f5-b0d1-788e1a894092\") " pod="openshift-dns/dns-default-b6zvm" Sep 09 22:59:32.722846 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.722822 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-image-registry-private-configuration\") pod \"image-registry-69855d874b-xvwj9\" (UID: \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\") " pod="openshift-image-registry/image-registry-69855d874b-xvwj9" Sep 09 22:59:32.722921 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.722892 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6251fed2-5675-42f5-b0d1-788e1a894092-config-volume\") pod \"dns-default-b6zvm\" (UID: \"6251fed2-5675-42f5-b0d1-788e1a894092\") " pod="openshift-dns/dns-default-b6zvm" Sep 09 22:59:32.723513 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.722924 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-installation-pull-secrets\") pod \"image-registry-69855d874b-xvwj9\" (UID: \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\") " pod="openshift-image-registry/image-registry-69855d874b-xvwj9" Sep 09 22:59:32.723513 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.722952 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8v2w\" (UniqueName: \"kubernetes.io/projected/6251fed2-5675-42f5-b0d1-788e1a894092-kube-api-access-z8v2w\") pod \"dns-default-b6zvm\" (UID: \"6251fed2-5675-42f5-b0d1-788e1a894092\") " pod="openshift-dns/dns-default-b6zvm" Sep 09 22:59:32.723513 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.723019 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6251fed2-5675-42f5-b0d1-788e1a894092-tmp-dir\") pod \"dns-default-b6zvm\" (UID: \"6251fed2-5675-42f5-b0d1-788e1a894092\") " pod="openshift-dns/dns-default-b6zvm" Sep 09 22:59:32.723513 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.723071 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-registry-certificates\") pod \"image-registry-69855d874b-xvwj9\" (UID: \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\") " pod="openshift-image-registry/image-registry-69855d874b-xvwj9" Sep 09 22:59:32.723513 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.723113 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6251fed2-5675-42f5-b0d1-788e1a894092-metrics-tls\") pod \"dns-default-b6zvm\" (UID: \"6251fed2-5675-42f5-b0d1-788e1a894092\") " pod="openshift-dns/dns-default-b6zvm" Sep 09 22:59:32.723513 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.723136 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-registry-tls\") pod \"image-registry-69855d874b-xvwj9\" (UID: \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\") " pod="openshift-image-registry/image-registry-69855d874b-xvwj9" Sep 09 22:59:32.723513 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.723158 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-bound-sa-token\") pod \"image-registry-69855d874b-xvwj9\" (UID: \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\") " pod="openshift-image-registry/image-registry-69855d874b-xvwj9" Sep 09 22:59:32.723513 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.723188 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5f001595-27ff-43d7-94d2-7ddc18d99671-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-zmvn9\" (UID: \"5f001595-27ff-43d7-94d2-7ddc18d99671\") " pod="openshift-network-console/networking-console-plugin-79977f9844-zmvn9" Sep 09 22:59:32.723513 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:32.723262 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 09 22:59:32.723513 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:32.723292 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 22:59:32.723513 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:32.723322 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f001595-27ff-43d7-94d2-7ddc18d99671-networking-console-plugin-cert podName:5f001595-27ff-43d7-94d2-7ddc18d99671 nodeName:}" failed. No retries permitted until 2026-09-09 22:59:33.223307252 +0000 UTC m=+34.573174264 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5f001595-27ff-43d7-94d2-7ddc18d99671-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-zmvn9" (UID: "5f001595-27ff-43d7-94d2-7ddc18d99671") : secret "networking-console-plugin-cert" not found Sep 09 22:59:32.723513 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:32.723266 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 22:59:32.723513 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:32.723363 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69855d874b-xvwj9: secret "image-registry-tls" not found Sep 09 22:59:32.723513 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:32.723347 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6251fed2-5675-42f5-b0d1-788e1a894092-metrics-tls podName:6251fed2-5675-42f5-b0d1-788e1a894092 nodeName:}" failed. No retries permitted until 2026-09-09 22:59:33.223331377 +0000 UTC m=+34.573198405 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6251fed2-5675-42f5-b0d1-788e1a894092-metrics-tls") pod "dns-default-b6zvm" (UID: "6251fed2-5675-42f5-b0d1-788e1a894092") : secret "dns-default-metrics-tls" not found Sep 09 22:59:32.723513 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.723406 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-289cb\" (UniqueName: \"kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-kube-api-access-289cb\") pod \"image-registry-69855d874b-xvwj9\" (UID: \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\") " pod="openshift-image-registry/image-registry-69855d874b-xvwj9" Sep 09 22:59:32.723513 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.723447 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1a033d1-6eb8-472d-95dc-5a1c26b60867-cert\") pod \"ingress-canary-5dl4c\" (UID: \"d1a033d1-6eb8-472d-95dc-5a1c26b60867\") " pod="openshift-ingress-canary/ingress-canary-5dl4c" Sep 09 22:59:32.724343 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:32.723461 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-registry-tls podName:c15d7152-f7f5-4d4c-8d66-4f955b5b9725 nodeName:}" failed. No retries permitted until 2026-09-09 22:59:33.223443261 +0000 UTC m=+34.573310277 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-registry-tls") pod "image-registry-69855d874b-xvwj9" (UID: "c15d7152-f7f5-4d4c-8d66-4f955b5b9725") : secret "image-registry-tls" not found Sep 09 22:59:32.724343 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.723471 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6251fed2-5675-42f5-b0d1-788e1a894092-config-volume\") pod \"dns-default-b6zvm\" (UID: \"6251fed2-5675-42f5-b0d1-788e1a894092\") " pod="openshift-dns/dns-default-b6zvm" Sep 09 22:59:32.724343 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.723532 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-trusted-ca\") pod \"image-registry-69855d874b-xvwj9\" (UID: \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\") " pod="openshift-image-registry/image-registry-69855d874b-xvwj9" Sep 09 22:59:32.724343 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.723541 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5f001595-27ff-43d7-94d2-7ddc18d99671-nginx-conf\") pod \"networking-console-plugin-79977f9844-zmvn9\" (UID: \"5f001595-27ff-43d7-94d2-7ddc18d99671\") " pod="openshift-network-console/networking-console-plugin-79977f9844-zmvn9" Sep 09 22:59:32.724343 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.723570 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-ca-trust-extracted\") pod \"image-registry-69855d874b-xvwj9\" (UID: \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\") " pod="openshift-image-registry/image-registry-69855d874b-xvwj9" Sep 09 22:59:32.724343 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.723865 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-ca-trust-extracted\") pod \"image-registry-69855d874b-xvwj9\" (UID: \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\") " pod="openshift-image-registry/image-registry-69855d874b-xvwj9" Sep 09 22:59:32.724343 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.724167 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5f001595-27ff-43d7-94d2-7ddc18d99671-nginx-conf\") pod \"networking-console-plugin-79977f9844-zmvn9\" (UID: \"5f001595-27ff-43d7-94d2-7ddc18d99671\") " pod="openshift-network-console/networking-console-plugin-79977f9844-zmvn9" Sep 09 22:59:32.727619 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.727599 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1a033d1-6eb8-472d-95dc-5a1c26b60867-cert\") pod \"ingress-canary-5dl4c\" (UID: \"d1a033d1-6eb8-472d-95dc-5a1c26b60867\") " pod="openshift-ingress-canary/ingress-canary-5dl4c" Sep 09 22:59:32.728038 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.728015 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-installation-pull-secrets\") pod \"image-registry-69855d874b-xvwj9\" (UID: \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\") " pod="openshift-image-registry/image-registry-69855d874b-xvwj9" Sep 09 22:59:32.728145 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.728056 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-image-registry-private-configuration\") pod \"image-registry-69855d874b-xvwj9\" (UID: \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\") " pod="openshift-image-registry/image-registry-69855d874b-xvwj9" Sep 09 22:59:32.731875 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.731851 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fz65j\" (UniqueName: \"kubernetes.io/projected/d1a033d1-6eb8-472d-95dc-5a1c26b60867-kube-api-access-fz65j\") pod \"ingress-canary-5dl4c\" (UID: \"d1a033d1-6eb8-472d-95dc-5a1c26b60867\") " pod="openshift-ingress-canary/ingress-canary-5dl4c" Sep 09 22:59:32.731957 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.731926 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-289cb\" (UniqueName: \"kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-kube-api-access-289cb\") pod \"image-registry-69855d874b-xvwj9\" (UID: \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\") " pod="openshift-image-registry/image-registry-69855d874b-xvwj9" Sep 09 22:59:32.732670 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.732631 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-bound-sa-token\") pod \"image-registry-69855d874b-xvwj9\" (UID: \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\") " pod="openshift-image-registry/image-registry-69855d874b-xvwj9" Sep 09 22:59:32.733066 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.733047 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8v2w\" (UniqueName: \"kubernetes.io/projected/6251fed2-5675-42f5-b0d1-788e1a894092-kube-api-access-z8v2w\") pod \"dns-default-b6zvm\" (UID: \"6251fed2-5675-42f5-b0d1-788e1a894092\") " pod="openshift-dns/dns-default-b6zvm" Sep 09 22:59:32.824793 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.824757 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cbe663c-0531-4339-b48b-c4d4c3dd5c2b-metrics-certs\") pod \"network-metrics-daemon-8dwkd\" (UID: \"7cbe663c-0531-4339-b48b-c4d4c3dd5c2b\") " pod="openshift-multus/network-metrics-daemon-8dwkd" Sep 09 22:59:32.824995 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:32.824921 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 22:59:32.825057 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:32.825016 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cbe663c-0531-4339-b48b-c4d4c3dd5c2b-metrics-certs podName:7cbe663c-0531-4339-b48b-c4d4c3dd5c2b nodeName:}" failed. No retries permitted until 2026-09-09 23:00:04.82499775 +0000 UTC m=+66.174864778 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7cbe663c-0531-4339-b48b-c4d4c3dd5c2b-metrics-certs") pod "network-metrics-daemon-8dwkd" (UID: "7cbe663c-0531-4339-b48b-c4d4c3dd5c2b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 22:59:32.828712 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.828689 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5dl4c" Sep 09 22:59:32.926341 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:32.926301 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/66786687-e3fe-42fd-8199-7f2a95232aa8-original-pull-secret\") pod \"global-pull-secret-syncer-b2gxt\" (UID: \"66786687-e3fe-42fd-8199-7f2a95232aa8\") " pod="kube-system/global-pull-secret-syncer-b2gxt" Sep 09 22:59:32.926516 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:32.926462 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 22:59:32.926573 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:32.926560 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66786687-e3fe-42fd-8199-7f2a95232aa8-original-pull-secret podName:66786687-e3fe-42fd-8199-7f2a95232aa8 nodeName:}" failed. No retries permitted until 2026-09-09 23:00:04.926543266 +0000 UTC m=+66.276410300 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/66786687-e3fe-42fd-8199-7f2a95232aa8-original-pull-secret") pod "global-pull-secret-syncer-b2gxt" (UID: "66786687-e3fe-42fd-8199-7f2a95232aa8") : object "kube-system"/"original-pull-secret" not registered Sep 09 22:59:33.026864 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:33.026819 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pd27g\" (UniqueName: \"kubernetes.io/projected/b85f7fcb-c8f2-43aa-9d28-890c87afcdd0-kube-api-access-pd27g\") pod \"network-check-target-gdhb9\" (UID: \"b85f7fcb-c8f2-43aa-9d28-890c87afcdd0\") " pod="openshift-network-diagnostics/network-check-target-gdhb9" Sep 09 22:59:33.027060 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:33.027019 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 22:59:33.027060 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:33.027042 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 22:59:33.027060 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:33.027056 2580 projected.go:196] Error preparing data for projected volume kube-api-access-pd27g for pod openshift-network-diagnostics/network-check-target-gdhb9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 22:59:33.027222 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:33.027120 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b85f7fcb-c8f2-43aa-9d28-890c87afcdd0-kube-api-access-pd27g podName:b85f7fcb-c8f2-43aa-9d28-890c87afcdd0 nodeName:}" failed. No retries permitted until 2026-09-09 23:00:05.02710166 +0000 UTC m=+66.376968676 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-pd27g" (UniqueName: "kubernetes.io/projected/b85f7fcb-c8f2-43aa-9d28-890c87afcdd0-kube-api-access-pd27g") pod "network-check-target-gdhb9" (UID: "b85f7fcb-c8f2-43aa-9d28-890c87afcdd0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 22:59:33.173498 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:33.173468 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gdhb9" Sep 09 22:59:33.173498 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:33.173503 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2gxt" Sep 09 22:59:33.174008 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:33.173468 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dwkd" Sep 09 22:59:33.176485 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:33.176431 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 09 22:59:33.176485 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:33.176448 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 09 22:59:33.176623 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:33.176511 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vjfnz\"" Sep 09 22:59:33.176623 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:33.176605 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 09 22:59:33.176772 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:33.176750 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 09 22:59:33.176895 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:33.176881 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-sns2d\"" Sep 09 22:59:33.228558 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:33.228521 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6251fed2-5675-42f5-b0d1-788e1a894092-metrics-tls\") pod \"dns-default-b6zvm\" (UID: \"6251fed2-5675-42f5-b0d1-788e1a894092\") " pod="openshift-dns/dns-default-b6zvm" Sep 09 22:59:33.228558 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:33.228561 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-registry-tls\") pod \"image-registry-69855d874b-xvwj9\" (UID: \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\") " pod="openshift-image-registry/image-registry-69855d874b-xvwj9" Sep 09 22:59:33.228778 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:33.228587 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5f001595-27ff-43d7-94d2-7ddc18d99671-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-zmvn9\" (UID: \"5f001595-27ff-43d7-94d2-7ddc18d99671\") " pod="openshift-network-console/networking-console-plugin-79977f9844-zmvn9" Sep 09 22:59:33.228778 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:33.228692 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 22:59:33.228778 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:33.228705 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 22:59:33.228778 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:33.228725 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69855d874b-xvwj9: secret "image-registry-tls" not found Sep 09 22:59:33.228778 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:33.228700 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 09 22:59:33.228778 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:33.228764 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6251fed2-5675-42f5-b0d1-788e1a894092-metrics-tls podName:6251fed2-5675-42f5-b0d1-788e1a894092 nodeName:}" failed. No retries permitted until 2026-09-09 22:59:34.228741602 +0000 UTC m=+35.578608638 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6251fed2-5675-42f5-b0d1-788e1a894092-metrics-tls") pod "dns-default-b6zvm" (UID: "6251fed2-5675-42f5-b0d1-788e1a894092") : secret "dns-default-metrics-tls" not found Sep 09 22:59:33.229083 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:33.228806 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-registry-tls podName:c15d7152-f7f5-4d4c-8d66-4f955b5b9725 nodeName:}" failed. No retries permitted until 2026-09-09 22:59:34.228788915 +0000 UTC m=+35.578655935 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-registry-tls") pod "image-registry-69855d874b-xvwj9" (UID: "c15d7152-f7f5-4d4c-8d66-4f955b5b9725") : secret "image-registry-tls" not found Sep 09 22:59:33.229083 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:33.228824 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f001595-27ff-43d7-94d2-7ddc18d99671-networking-console-plugin-cert podName:5f001595-27ff-43d7-94d2-7ddc18d99671 nodeName:}" failed. No retries permitted until 2026-09-09 22:59:34.228818218 +0000 UTC m=+35.578685229 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5f001595-27ff-43d7-94d2-7ddc18d99671-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-zmvn9" (UID: "5f001595-27ff-43d7-94d2-7ddc18d99671") : secret "networking-console-plugin-cert" not found Sep 09 22:59:34.087883 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:34.087757 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5dl4c"] Sep 09 22:59:34.091148 ip-10-0-131-92 kubenswrapper[2580]: W0909 22:59:34.091109 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1a033d1_6eb8_472d_95dc_5a1c26b60867.slice/crio-a9b29f705d1c9fa97807d6adf1c2f55a8748e6f521fa3376ce6b0a19a713db4e WatchSource:0}: Error finding container a9b29f705d1c9fa97807d6adf1c2f55a8748e6f521fa3376ce6b0a19a713db4e: Status 404 returned error can't find the container with id a9b29f705d1c9fa97807d6adf1c2f55a8748e6f521fa3376ce6b0a19a713db4e Sep 09 22:59:34.186667 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:34.186634 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ps2b2_2a2906d3-5dc1-441e-b7f8-a97e2a67d4ed/konnectivity-agent/0.log" limit=1024 Sep 09 22:59:34.236174 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:34.236030 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6251fed2-5675-42f5-b0d1-788e1a894092-metrics-tls\") pod \"dns-default-b6zvm\" (UID: \"6251fed2-5675-42f5-b0d1-788e1a894092\") " pod="openshift-dns/dns-default-b6zvm" Sep 09 22:59:34.236313 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:34.236170 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 22:59:34.236313 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:34.236187 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-registry-tls\") pod \"image-registry-69855d874b-xvwj9\" (UID: \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\") " pod="openshift-image-registry/image-registry-69855d874b-xvwj9" Sep 09 22:59:34.236313 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:34.236227 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6251fed2-5675-42f5-b0d1-788e1a894092-metrics-tls podName:6251fed2-5675-42f5-b0d1-788e1a894092 nodeName:}" failed. No retries permitted until 2026-09-09 22:59:36.236211593 +0000 UTC m=+37.586078605 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6251fed2-5675-42f5-b0d1-788e1a894092-metrics-tls") pod "dns-default-b6zvm" (UID: "6251fed2-5675-42f5-b0d1-788e1a894092") : secret "dns-default-metrics-tls" not found Sep 09 22:59:34.236313 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:34.236246 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5f001595-27ff-43d7-94d2-7ddc18d99671-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-zmvn9\" (UID: \"5f001595-27ff-43d7-94d2-7ddc18d99671\") " pod="openshift-network-console/networking-console-plugin-79977f9844-zmvn9" Sep 09 22:59:34.236313 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:34.236288 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 22:59:34.236313 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:34.236301 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69855d874b-xvwj9: secret "image-registry-tls" not found Sep 09 22:59:34.236523 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:34.236345 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-registry-tls podName:c15d7152-f7f5-4d4c-8d66-4f955b5b9725 nodeName:}" failed. No retries permitted until 2026-09-09 22:59:36.236330347 +0000 UTC m=+37.586197374 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-registry-tls") pod "image-registry-69855d874b-xvwj9" (UID: "c15d7152-f7f5-4d4c-8d66-4f955b5b9725") : secret "image-registry-tls" not found Sep 09 22:59:34.236523 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:34.236377 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 09 22:59:34.236523 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:34.236443 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f001595-27ff-43d7-94d2-7ddc18d99671-networking-console-plugin-cert podName:5f001595-27ff-43d7-94d2-7ddc18d99671 nodeName:}" failed. No retries permitted until 2026-09-09 22:59:36.236429114 +0000 UTC m=+37.586296141 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5f001595-27ff-43d7-94d2-7ddc18d99671-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-zmvn9" (UID: "5f001595-27ff-43d7-94d2-7ddc18d99671") : secret "networking-console-plugin-cert" not found Sep 09 22:59:34.424727 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:34.424693 2580 generic.go:356] "Generic (PLEG): container finished" podID="09c46a31-75e9-40a5-856f-4f53ae4b0a3b" containerID="f4672290e130dd14471d52e57c42fce7bcaa57a14df4f254fe84af19d5cd319b" exitCode=0 Sep 09 22:59:34.424865 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:34.424769 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dc58c" event={"ID":"09c46a31-75e9-40a5-856f-4f53ae4b0a3b","Type":"ContainerDied","Data":"f4672290e130dd14471d52e57c42fce7bcaa57a14df4f254fe84af19d5cd319b"} Sep 09 22:59:34.425857 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:34.425836 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5dl4c" event={"ID":"d1a033d1-6eb8-472d-95dc-5a1c26b60867","Type":"ContainerStarted","Data":"a9b29f705d1c9fa97807d6adf1c2f55a8748e6f521fa3376ce6b0a19a713db4e"} Sep 09 22:59:35.432461 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:35.432425 2580 generic.go:356] "Generic (PLEG): container finished" podID="09c46a31-75e9-40a5-856f-4f53ae4b0a3b" containerID="722fdcb85aab08915b0e6a5a9af990fd0ab1b26a2a21883f1cbce0e212cf4ffd" exitCode=0 Sep 09 22:59:35.432924 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:35.432506 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dc58c" event={"ID":"09c46a31-75e9-40a5-856f-4f53ae4b0a3b","Type":"ContainerDied","Data":"722fdcb85aab08915b0e6a5a9af990fd0ab1b26a2a21883f1cbce0e212cf4ffd"} Sep 09 22:59:36.253176 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:36.253142 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6251fed2-5675-42f5-b0d1-788e1a894092-metrics-tls\") pod \"dns-default-b6zvm\" (UID: \"6251fed2-5675-42f5-b0d1-788e1a894092\") " pod="openshift-dns/dns-default-b6zvm" Sep 09 22:59:36.253176 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:36.253178 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-registry-tls\") pod \"image-registry-69855d874b-xvwj9\" (UID: \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\") " pod="openshift-image-registry/image-registry-69855d874b-xvwj9" Sep 09 22:59:36.253383 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:36.253213 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5f001595-27ff-43d7-94d2-7ddc18d99671-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-zmvn9\" (UID: \"5f001595-27ff-43d7-94d2-7ddc18d99671\") " pod="openshift-network-console/networking-console-plugin-79977f9844-zmvn9" Sep 09 22:59:36.253383 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:36.253313 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 22:59:36.253383 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:36.253327 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 09 22:59:36.253383 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:36.253319 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 22:59:36.253383 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:36.253376 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69855d874b-xvwj9: secret "image-registry-tls" not found Sep 09 22:59:36.253383 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:36.253381 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f001595-27ff-43d7-94d2-7ddc18d99671-networking-console-plugin-cert podName:5f001595-27ff-43d7-94d2-7ddc18d99671 nodeName:}" failed. No retries permitted until 2026-09-09 22:59:40.253363597 +0000 UTC m=+41.603230608 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5f001595-27ff-43d7-94d2-7ddc18d99671-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-zmvn9" (UID: "5f001595-27ff-43d7-94d2-7ddc18d99671") : secret "networking-console-plugin-cert" not found Sep 09 22:59:36.253591 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:36.253394 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6251fed2-5675-42f5-b0d1-788e1a894092-metrics-tls podName:6251fed2-5675-42f5-b0d1-788e1a894092 nodeName:}" failed. No retries permitted until 2026-09-09 22:59:40.253388236 +0000 UTC m=+41.603255248 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6251fed2-5675-42f5-b0d1-788e1a894092-metrics-tls") pod "dns-default-b6zvm" (UID: "6251fed2-5675-42f5-b0d1-788e1a894092") : secret "dns-default-metrics-tls" not found Sep 09 22:59:36.253591 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:36.253415 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-registry-tls podName:c15d7152-f7f5-4d4c-8d66-4f955b5b9725 nodeName:}" failed. No retries permitted until 2026-09-09 22:59:40.253398623 +0000 UTC m=+41.603265653 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-registry-tls") pod "image-registry-69855d874b-xvwj9" (UID: "c15d7152-f7f5-4d4c-8d66-4f955b5b9725") : secret "image-registry-tls" not found Sep 09 22:59:36.440301 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:36.440252 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dc58c" event={"ID":"09c46a31-75e9-40a5-856f-4f53ae4b0a3b","Type":"ContainerStarted","Data":"cbc6107b2b6a723141de50ec5da9586ecb4f800fa4d81076d012a26e97f18075"} Sep 09 22:59:36.464066 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:36.464007 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-dc58c" podStartSLOduration=8.31839724 podStartE2EDuration="37.463986591s" totalImagesPullingTime="29.145589351s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 22:58:59 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 22:59:36.462017504 +0000 UTC m=+37.811884538" watchObservedRunningTime="2026-09-09 22:59:36.463986591 +0000 UTC m=+37.813853621" Sep 09 22:59:36.582960 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:36.582881 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ps2b2_2a2906d3-5dc1-441e-b7f8-a97e2a67d4ed/konnectivity-agent/0.log" limit=1024 Sep 09 22:59:36.683313 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:36.683278 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ps2b2_2a2906d3-5dc1-441e-b7f8-a97e2a67d4ed/konnectivity-agent/0.log" limit=1024 Sep 09 22:59:37.444564 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:37.444530 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5dl4c" event={"ID":"d1a033d1-6eb8-472d-95dc-5a1c26b60867","Type":"ContainerStarted","Data":"0d17e81210eeac2c9879df9ce8edbb04a91472548250c9bc6083d89a95c87501"} Sep 09 22:59:37.462358 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:37.462308 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5dl4c" podStartSLOduration=2.565090342 podStartE2EDuration="5.462293563s" totalImagesPullingTime="2.897203221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 22:59:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 22:59:37.461716483 +0000 UTC m=+38.811583519" watchObservedRunningTime="2026-09-09 22:59:37.462293563 +0000 UTC m=+38.812160615" Sep 09 22:59:38.024915 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:38.024884 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ps2b2_2a2906d3-5dc1-441e-b7f8-a97e2a67d4ed/konnectivity-agent/0.log" limit=1024 Sep 09 22:59:38.078235 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:38.078206 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ps2b2_2a2906d3-5dc1-441e-b7f8-a97e2a67d4ed/konnectivity-agent/0.log" limit=1024 Sep 09 22:59:38.226913 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:38.226882 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ps2b2_2a2906d3-5dc1-441e-b7f8-a97e2a67d4ed/konnectivity-agent/0.log" limit=1024 Sep 09 22:59:38.327306 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:38.327229 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ps2b2_2a2906d3-5dc1-441e-b7f8-a97e2a67d4ed/konnectivity-agent/0.log" limit=1024 Sep 09 22:59:39.310258 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:39.310228 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ps2b2_2a2906d3-5dc1-441e-b7f8-a97e2a67d4ed/konnectivity-agent/0.log" limit=1024 Sep 09 22:59:39.358456 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:39.358428 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ps2b2_2a2906d3-5dc1-441e-b7f8-a97e2a67d4ed/konnectivity-agent/0.log" limit=1024 Sep 09 22:59:40.284124 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:40.284088 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5f001595-27ff-43d7-94d2-7ddc18d99671-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-zmvn9\" (UID: \"5f001595-27ff-43d7-94d2-7ddc18d99671\") " pod="openshift-network-console/networking-console-plugin-79977f9844-zmvn9" Sep 09 22:59:40.284277 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:40.284202 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6251fed2-5675-42f5-b0d1-788e1a894092-metrics-tls\") pod \"dns-default-b6zvm\" (UID: \"6251fed2-5675-42f5-b0d1-788e1a894092\") " pod="openshift-dns/dns-default-b6zvm" Sep 09 22:59:40.284277 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:40.284244 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 09 22:59:40.284277 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:40.284233 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-registry-tls\") pod \"image-registry-69855d874b-xvwj9\" (UID: \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\") " pod="openshift-image-registry/image-registry-69855d874b-xvwj9" Sep 09 22:59:40.284432 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:40.284311 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f001595-27ff-43d7-94d2-7ddc18d99671-networking-console-plugin-cert podName:5f001595-27ff-43d7-94d2-7ddc18d99671 nodeName:}" failed. No retries permitted until 2026-09-09 22:59:48.284292729 +0000 UTC m=+49.634159744 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5f001595-27ff-43d7-94d2-7ddc18d99671-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-zmvn9" (UID: "5f001595-27ff-43d7-94d2-7ddc18d99671") : secret "networking-console-plugin-cert" not found Sep 09 22:59:40.284432 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:40.284349 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 22:59:40.284432 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:40.284363 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69855d874b-xvwj9: secret "image-registry-tls" not found Sep 09 22:59:40.284432 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:40.284370 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 22:59:40.284432 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:40.284404 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-registry-tls podName:c15d7152-f7f5-4d4c-8d66-4f955b5b9725 nodeName:}" failed. No retries permitted until 2026-09-09 22:59:48.284392036 +0000 UTC m=+49.634259061 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-registry-tls") pod "image-registry-69855d874b-xvwj9" (UID: "c15d7152-f7f5-4d4c-8d66-4f955b5b9725") : secret "image-registry-tls" not found Sep 09 22:59:40.284599 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:40.284433 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6251fed2-5675-42f5-b0d1-788e1a894092-metrics-tls podName:6251fed2-5675-42f5-b0d1-788e1a894092 nodeName:}" failed. No retries permitted until 2026-09-09 22:59:48.284415757 +0000 UTC m=+49.634282768 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6251fed2-5675-42f5-b0d1-788e1a894092-metrics-tls") pod "dns-default-b6zvm" (UID: "6251fed2-5675-42f5-b0d1-788e1a894092") : secret "dns-default-metrics-tls" not found Sep 09 22:59:41.700482 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:41.700430 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ps2b2_2a2906d3-5dc1-441e-b7f8-a97e2a67d4ed/konnectivity-agent/0.log" limit=1024 Sep 09 22:59:41.758621 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:41.758582 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ps2b2_2a2906d3-5dc1-441e-b7f8-a97e2a67d4ed/konnectivity-agent/0.log" limit=1024 Sep 09 22:59:48.342755 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:48.342714 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6251fed2-5675-42f5-b0d1-788e1a894092-metrics-tls\") pod \"dns-default-b6zvm\" (UID: \"6251fed2-5675-42f5-b0d1-788e1a894092\") " pod="openshift-dns/dns-default-b6zvm" Sep 09 22:59:48.342755 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:48.342757 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-registry-tls\") pod \"image-registry-69855d874b-xvwj9\" (UID: \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\") " pod="openshift-image-registry/image-registry-69855d874b-xvwj9" Sep 09 22:59:48.343242 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:48.342785 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5f001595-27ff-43d7-94d2-7ddc18d99671-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-zmvn9\" (UID: \"5f001595-27ff-43d7-94d2-7ddc18d99671\") " pod="openshift-network-console/networking-console-plugin-79977f9844-zmvn9" Sep 09 22:59:48.343242 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:48.342863 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 22:59:48.343242 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:48.342881 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 22:59:48.343242 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:48.342894 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69855d874b-xvwj9: secret "image-registry-tls" not found Sep 09 22:59:48.343242 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:48.342927 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6251fed2-5675-42f5-b0d1-788e1a894092-metrics-tls podName:6251fed2-5675-42f5-b0d1-788e1a894092 nodeName:}" failed. No retries permitted until 2026-09-09 23:00:04.342911188 +0000 UTC m=+65.692778199 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6251fed2-5675-42f5-b0d1-788e1a894092-metrics-tls") pod "dns-default-b6zvm" (UID: "6251fed2-5675-42f5-b0d1-788e1a894092") : secret "dns-default-metrics-tls" not found Sep 09 22:59:48.343242 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:48.342942 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-registry-tls podName:c15d7152-f7f5-4d4c-8d66-4f955b5b9725 nodeName:}" failed. No retries permitted until 2026-09-09 23:00:04.34293667 +0000 UTC m=+65.692803683 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-registry-tls") pod "image-registry-69855d874b-xvwj9" (UID: "c15d7152-f7f5-4d4c-8d66-4f955b5b9725") : secret "image-registry-tls" not found Sep 09 22:59:48.343242 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:48.342940 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 09 22:59:48.343242 ip-10-0-131-92 kubenswrapper[2580]: E0909 22:59:48.342968 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f001595-27ff-43d7-94d2-7ddc18d99671-networking-console-plugin-cert podName:5f001595-27ff-43d7-94d2-7ddc18d99671 nodeName:}" failed. No retries permitted until 2026-09-09 23:00:04.342961994 +0000 UTC m=+65.692829007 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5f001595-27ff-43d7-94d2-7ddc18d99671-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-zmvn9" (UID: "5f001595-27ff-43d7-94d2-7ddc18d99671") : secret "networking-console-plugin-cert" not found Sep 09 22:59:58.414954 ip-10-0-131-92 kubenswrapper[2580]: I0909 22:59:58.414915 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-prbnz" Sep 09 23:00:04.366153 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:00:04.366104 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6251fed2-5675-42f5-b0d1-788e1a894092-metrics-tls\") pod \"dns-default-b6zvm\" (UID: \"6251fed2-5675-42f5-b0d1-788e1a894092\") " pod="openshift-dns/dns-default-b6zvm" Sep 09 23:00:04.366153 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:00:04.366154 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-registry-tls\") pod \"image-registry-69855d874b-xvwj9\" (UID: \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\") " pod="openshift-image-registry/image-registry-69855d874b-xvwj9" Sep 09 23:00:04.366691 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:00:04.366184 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5f001595-27ff-43d7-94d2-7ddc18d99671-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-zmvn9\" (UID: \"5f001595-27ff-43d7-94d2-7ddc18d99671\") " pod="openshift-network-console/networking-console-plugin-79977f9844-zmvn9" Sep 09 23:00:04.366691 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:00:04.366254 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 23:00:04.366691 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:00:04.366279 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 23:00:04.366691 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:00:04.366298 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 09 23:00:04.366691 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:00:04.366325 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6251fed2-5675-42f5-b0d1-788e1a894092-metrics-tls podName:6251fed2-5675-42f5-b0d1-788e1a894092 nodeName:}" failed. No retries permitted until 2026-09-09 23:00:36.366309892 +0000 UTC m=+97.716176903 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6251fed2-5675-42f5-b0d1-788e1a894092-metrics-tls") pod "dns-default-b6zvm" (UID: "6251fed2-5675-42f5-b0d1-788e1a894092") : secret "dns-default-metrics-tls" not found Sep 09 23:00:04.366691 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:00:04.366340 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f001595-27ff-43d7-94d2-7ddc18d99671-networking-console-plugin-cert podName:5f001595-27ff-43d7-94d2-7ddc18d99671 nodeName:}" failed. No retries permitted until 2026-09-09 23:00:36.366334005 +0000 UTC m=+97.716201017 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5f001595-27ff-43d7-94d2-7ddc18d99671-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-zmvn9" (UID: "5f001595-27ff-43d7-94d2-7ddc18d99671") : secret "networking-console-plugin-cert" not found Sep 09 23:00:04.366691 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:00:04.366298 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69855d874b-xvwj9: secret "image-registry-tls" not found Sep 09 23:00:04.366691 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:00:04.366420 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-registry-tls podName:c15d7152-f7f5-4d4c-8d66-4f955b5b9725 nodeName:}" failed. No retries permitted until 2026-09-09 23:00:36.366404572 +0000 UTC m=+97.716271589 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-registry-tls") pod "image-registry-69855d874b-xvwj9" (UID: "c15d7152-f7f5-4d4c-8d66-4f955b5b9725") : secret "image-registry-tls" not found Sep 09 23:00:04.869514 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:00:04.869472 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cbe663c-0531-4339-b48b-c4d4c3dd5c2b-metrics-certs\") pod \"network-metrics-daemon-8dwkd\" (UID: \"7cbe663c-0531-4339-b48b-c4d4c3dd5c2b\") " pod="openshift-multus/network-metrics-daemon-8dwkd" Sep 09 23:00:04.872306 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:00:04.872287 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 09 23:00:04.880425 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:00:04.880404 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 09 23:00:04.880481 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:00:04.880469 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cbe663c-0531-4339-b48b-c4d4c3dd5c2b-metrics-certs podName:7cbe663c-0531-4339-b48b-c4d4c3dd5c2b nodeName:}" failed. No retries permitted until 2026-09-09 23:01:08.880447999 +0000 UTC m=+130.230315024 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7cbe663c-0531-4339-b48b-c4d4c3dd5c2b-metrics-certs") pod "network-metrics-daemon-8dwkd" (UID: "7cbe663c-0531-4339-b48b-c4d4c3dd5c2b") : secret "metrics-daemon-secret" not found Sep 09 23:00:04.970711 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:00:04.970679 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/66786687-e3fe-42fd-8199-7f2a95232aa8-original-pull-secret\") pod \"global-pull-secret-syncer-b2gxt\" (UID: \"66786687-e3fe-42fd-8199-7f2a95232aa8\") " pod="kube-system/global-pull-secret-syncer-b2gxt" Sep 09 23:00:04.973750 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:00:04.973730 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 09 23:00:04.984304 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:00:04.984277 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/66786687-e3fe-42fd-8199-7f2a95232aa8-original-pull-secret\") pod \"global-pull-secret-syncer-b2gxt\" (UID: \"66786687-e3fe-42fd-8199-7f2a95232aa8\") " pod="kube-system/global-pull-secret-syncer-b2gxt" Sep 09 23:00:04.994463 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:00:04.994438 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2gxt" Sep 09 23:00:05.071563 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:00:05.071535 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pd27g\" (UniqueName: \"kubernetes.io/projected/b85f7fcb-c8f2-43aa-9d28-890c87afcdd0-kube-api-access-pd27g\") pod \"network-check-target-gdhb9\" (UID: \"b85f7fcb-c8f2-43aa-9d28-890c87afcdd0\") " pod="openshift-network-diagnostics/network-check-target-gdhb9" Sep 09 23:00:05.074341 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:00:05.074323 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 09 23:00:05.084788 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:00:05.084767 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 09 23:00:05.095115 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:00:05.095093 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pd27g\" (UniqueName: \"kubernetes.io/projected/b85f7fcb-c8f2-43aa-9d28-890c87afcdd0-kube-api-access-pd27g\") pod \"network-check-target-gdhb9\" (UID: \"b85f7fcb-c8f2-43aa-9d28-890c87afcdd0\") " pod="openshift-network-diagnostics/network-check-target-gdhb9" Sep 09 23:00:05.266872 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:00:05.266835 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-b2gxt"] Sep 09 23:00:05.269692 ip-10-0-131-92 kubenswrapper[2580]: W0909 23:00:05.269662 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66786687_e3fe_42fd_8199_7f2a95232aa8.slice/crio-2612b3f84c9edfbb862f3b60a3fca4cb1ae44c0216e49c8b323b3ccda351db99 WatchSource:0}: Error finding container 2612b3f84c9edfbb862f3b60a3fca4cb1ae44c0216e49c8b323b3ccda351db99: Status 404 returned error can't find the container with id 2612b3f84c9edfbb862f3b60a3fca4cb1ae44c0216e49c8b323b3ccda351db99 Sep 09 23:00:05.290060 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:00:05.290034 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-sns2d\"" Sep 09 23:00:05.298328 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:00:05.298309 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gdhb9" Sep 09 23:00:05.509005 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:00:05.508952 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-b2gxt" event={"ID":"66786687-e3fe-42fd-8199-7f2a95232aa8","Type":"ContainerStarted","Data":"2612b3f84c9edfbb862f3b60a3fca4cb1ae44c0216e49c8b323b3ccda351db99"} Sep 09 23:00:05.565324 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:00:05.565247 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gdhb9"] Sep 09 23:00:05.568203 ip-10-0-131-92 kubenswrapper[2580]: W0909 23:00:05.568177 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb85f7fcb_c8f2_43aa_9d28_890c87afcdd0.slice/crio-6b444f11f478364ba39b1155e6abd48d677f919f3d094e2732102a8bacd292f6 WatchSource:0}: Error finding container 6b444f11f478364ba39b1155e6abd48d677f919f3d094e2732102a8bacd292f6: Status 404 returned error can't find the container with id 6b444f11f478364ba39b1155e6abd48d677f919f3d094e2732102a8bacd292f6 Sep 09 23:00:06.513678 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:00:06.513638 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gdhb9" event={"ID":"b85f7fcb-c8f2-43aa-9d28-890c87afcdd0","Type":"ContainerStarted","Data":"6b444f11f478364ba39b1155e6abd48d677f919f3d094e2732102a8bacd292f6"} Sep 09 23:00:11.528344 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:00:11.528307 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-b2gxt" event={"ID":"66786687-e3fe-42fd-8199-7f2a95232aa8","Type":"ContainerStarted","Data":"46cfacd4c909fa2aae405a82b9754aef61c1b2610ef6b5d5bf2bd8cc9b75932b"} Sep 09 23:00:11.529772 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:00:11.529742 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gdhb9" event={"ID":"b85f7fcb-c8f2-43aa-9d28-890c87afcdd0","Type":"ContainerStarted","Data":"e142102a05c376f458975d0a70fd26b407c3d2b83c8277bc660e535493461835"} Sep 09 23:00:11.529885 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:00:11.529871 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-gdhb9" Sep 09 23:00:11.544511 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:00:11.544470 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-b2gxt" podStartSLOduration=66.826340247 podStartE2EDuration="1m12.544457183s" totalImagesPullingTime="5.718116936s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 22:58:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:00:11.543359013 +0000 UTC m=+72.893226048" watchObservedRunningTime="2026-09-09 23:00:11.544457183 +0000 UTC m=+72.894324217" Sep 09 23:00:11.557343 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:00:11.557298 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-gdhb9" podStartSLOduration=67.137760155 podStartE2EDuration="1m12.557285161s" totalImagesPullingTime="5.419525006s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 22:58:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:00:11.556886994 +0000 UTC m=+72.906754029" watchObservedRunningTime="2026-09-09 23:00:11.557285161 +0000 UTC m=+72.907152196" Sep 09 23:00:36.385529 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:00:36.385403 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6251fed2-5675-42f5-b0d1-788e1a894092-metrics-tls\") pod \"dns-default-b6zvm\" (UID: \"6251fed2-5675-42f5-b0d1-788e1a894092\") " pod="openshift-dns/dns-default-b6zvm" Sep 09 23:00:36.385529 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:00:36.385445 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-registry-tls\") pod \"image-registry-69855d874b-xvwj9\" (UID: \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\") " pod="openshift-image-registry/image-registry-69855d874b-xvwj9" Sep 09 23:00:36.385529 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:00:36.385471 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5f001595-27ff-43d7-94d2-7ddc18d99671-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-zmvn9\" (UID: \"5f001595-27ff-43d7-94d2-7ddc18d99671\") " pod="openshift-network-console/networking-console-plugin-79977f9844-zmvn9" Sep 09 23:00:36.386072 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:00:36.385578 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 09 23:00:36.386072 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:00:36.385598 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 23:00:36.386072 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:00:36.385617 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69855d874b-xvwj9: secret "image-registry-tls" not found Sep 09 23:00:36.386072 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:00:36.385638 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f001595-27ff-43d7-94d2-7ddc18d99671-networking-console-plugin-cert podName:5f001595-27ff-43d7-94d2-7ddc18d99671 nodeName:}" failed. No retries permitted until 2026-09-09 23:01:40.385624027 +0000 UTC m=+161.735491039 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5f001595-27ff-43d7-94d2-7ddc18d99671-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-zmvn9" (UID: "5f001595-27ff-43d7-94d2-7ddc18d99671") : secret "networking-console-plugin-cert" not found Sep 09 23:00:36.386072 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:00:36.385665 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-registry-tls podName:c15d7152-f7f5-4d4c-8d66-4f955b5b9725 nodeName:}" failed. No retries permitted until 2026-09-09 23:01:40.385653026 +0000 UTC m=+161.735520037 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-registry-tls") pod "image-registry-69855d874b-xvwj9" (UID: "c15d7152-f7f5-4d4c-8d66-4f955b5b9725") : secret "image-registry-tls" not found Sep 09 23:00:36.386072 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:00:36.385578 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 23:00:36.386072 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:00:36.385713 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6251fed2-5675-42f5-b0d1-788e1a894092-metrics-tls podName:6251fed2-5675-42f5-b0d1-788e1a894092 nodeName:}" failed. No retries permitted until 2026-09-09 23:01:40.385701314 +0000 UTC m=+161.735568344 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6251fed2-5675-42f5-b0d1-788e1a894092-metrics-tls") pod "dns-default-b6zvm" (UID: "6251fed2-5675-42f5-b0d1-788e1a894092") : secret "dns-default-metrics-tls" not found Sep 09 23:00:42.534420 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:00:42.534390 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-gdhb9" Sep 09 23:01:08.924295 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:01:08.924254 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cbe663c-0531-4339-b48b-c4d4c3dd5c2b-metrics-certs\") pod \"network-metrics-daemon-8dwkd\" (UID: \"7cbe663c-0531-4339-b48b-c4d4c3dd5c2b\") " pod="openshift-multus/network-metrics-daemon-8dwkd" Sep 09 23:01:08.924806 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:01:08.924386 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 09 23:01:08.924806 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:01:08.924474 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cbe663c-0531-4339-b48b-c4d4c3dd5c2b-metrics-certs podName:7cbe663c-0531-4339-b48b-c4d4c3dd5c2b nodeName:}" failed. No retries permitted until 2026-09-09 23:03:10.924449696 +0000 UTC m=+252.274316710 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7cbe663c-0531-4339-b48b-c4d4c3dd5c2b-metrics-certs") pod "network-metrics-daemon-8dwkd" (UID: "7cbe663c-0531-4339-b48b-c4d4c3dd5c2b") : secret "metrics-daemon-secret" not found Sep 09 23:01:35.492808 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:01:35.492748 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-69855d874b-xvwj9" podUID="c15d7152-f7f5-4d4c-8d66-4f955b5b9725" Sep 09 23:01:35.503950 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:01:35.503918 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-79977f9844-zmvn9" podUID="5f001595-27ff-43d7-94d2-7ddc18d99671" Sep 09 23:01:35.515290 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:01:35.515263 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-b6zvm" podUID="6251fed2-5675-42f5-b0d1-788e1a894092" Sep 09 23:01:35.731154 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:01:35.731126 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69855d874b-xvwj9" Sep 09 23:01:35.731292 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:01:35.731126 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b6zvm" Sep 09 23:01:36.199903 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:01:36.199855 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-8dwkd" podUID="7cbe663c-0531-4339-b48b-c4d4c3dd5c2b" Sep 09 23:01:40.443483 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:01:40.443417 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6251fed2-5675-42f5-b0d1-788e1a894092-metrics-tls\") pod \"dns-default-b6zvm\" (UID: \"6251fed2-5675-42f5-b0d1-788e1a894092\") " pod="openshift-dns/dns-default-b6zvm" Sep 09 23:01:40.443483 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:01:40.443482 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-registry-tls\") pod \"image-registry-69855d874b-xvwj9\" (UID: \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\") " pod="openshift-image-registry/image-registry-69855d874b-xvwj9" Sep 09 23:01:40.443922 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:01:40.443520 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5f001595-27ff-43d7-94d2-7ddc18d99671-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-zmvn9\" (UID: \"5f001595-27ff-43d7-94d2-7ddc18d99671\") " pod="openshift-network-console/networking-console-plugin-79977f9844-zmvn9" Sep 09 23:01:40.443922 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:01:40.443587 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 23:01:40.443922 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:01:40.443615 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 09 23:01:40.443922 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:01:40.443646 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 23:01:40.443922 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:01:40.443667 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69855d874b-xvwj9: secret "image-registry-tls" not found Sep 09 23:01:40.443922 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:01:40.443659 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6251fed2-5675-42f5-b0d1-788e1a894092-metrics-tls podName:6251fed2-5675-42f5-b0d1-788e1a894092 nodeName:}" failed. No retries permitted until 2026-09-09 23:03:42.443639712 +0000 UTC m=+283.793506737 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6251fed2-5675-42f5-b0d1-788e1a894092-metrics-tls") pod "dns-default-b6zvm" (UID: "6251fed2-5675-42f5-b0d1-788e1a894092") : secret "dns-default-metrics-tls" not found Sep 09 23:01:40.443922 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:01:40.443742 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f001595-27ff-43d7-94d2-7ddc18d99671-networking-console-plugin-cert podName:5f001595-27ff-43d7-94d2-7ddc18d99671 nodeName:}" failed. No retries permitted until 2026-09-09 23:03:42.443723257 +0000 UTC m=+283.793590273 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5f001595-27ff-43d7-94d2-7ddc18d99671-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-zmvn9" (UID: "5f001595-27ff-43d7-94d2-7ddc18d99671") : secret "networking-console-plugin-cert" not found Sep 09 23:01:40.443922 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:01:40.443760 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-registry-tls podName:c15d7152-f7f5-4d4c-8d66-4f955b5b9725 nodeName:}" failed. No retries permitted until 2026-09-09 23:03:42.443753281 +0000 UTC m=+283.793620292 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-registry-tls") pod "image-registry-69855d874b-xvwj9" (UID: "c15d7152-f7f5-4d4c-8d66-4f955b5b9725") : secret "image-registry-tls" not found Sep 09 23:01:50.172931 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:01:50.172880 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-zmvn9" Sep 09 23:01:51.174004 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:01:51.173907 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dwkd" Sep 09 23:03:10.984477 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:10.984432 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cbe663c-0531-4339-b48b-c4d4c3dd5c2b-metrics-certs\") pod \"network-metrics-daemon-8dwkd\" (UID: \"7cbe663c-0531-4339-b48b-c4d4c3dd5c2b\") " pod="openshift-multus/network-metrics-daemon-8dwkd" Sep 09 23:03:10.984958 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:10.984576 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 09 23:03:10.984958 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:10.984641 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cbe663c-0531-4339-b48b-c4d4c3dd5c2b-metrics-certs podName:7cbe663c-0531-4339-b48b-c4d4c3dd5c2b nodeName:}" failed. No retries permitted until 2026-09-09 23:05:12.984626852 +0000 UTC m=+374.334493865 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7cbe663c-0531-4339-b48b-c4d4c3dd5c2b-metrics-certs") pod "network-metrics-daemon-8dwkd" (UID: "7cbe663c-0531-4339-b48b-c4d4c3dd5c2b") : secret "metrics-daemon-secret" not found Sep 09 23:03:18.154401 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.154359 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-qctkt"] Sep 09 23:03:18.159489 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.159459 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qctkt" Sep 09 23:03:18.160689 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.160666 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dtbbq"] Sep 09 23:03:18.162549 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.162522 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Sep 09 23:03:18.163380 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.163355 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Sep 09 23:03:18.163472 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.163386 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Sep 09 23:03:18.163472 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.163362 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-fqrvg\"" Sep 09 23:03:18.164020 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.164003 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Sep 09 23:03:18.164216 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.164200 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-4p2j4"] Sep 09 23:03:18.164334 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.164320 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dtbbq" Sep 09 23:03:18.167040 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.167021 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-xht2x"] Sep 09 23:03:18.167129 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.167114 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-4p2j4" Sep 09 23:03:18.169694 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.169675 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-xht2x" Sep 09 23:03:18.169968 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.169945 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Sep 09 23:03:18.170081 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.169985 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Sep 09 23:03:18.170533 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.170357 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Sep 09 23:03:18.170533 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.170373 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Sep 09 23:03:18.170533 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.170418 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-c2g95\"" Sep 09 23:03:18.170771 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.170755 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Sep 09 23:03:18.170956 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.170939 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Sep 09 23:03:18.171064 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.171046 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Sep 09 23:03:18.171173 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.171103 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-qctkt"] Sep 09 23:03:18.171391 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.171377 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-sfs7k\"" Sep 09 23:03:18.172345 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.172327 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Sep 09 23:03:18.173712 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.173622 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Sep 09 23:03:18.173952 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.173928 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-gcwdg\"" Sep 09 23:03:18.174472 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.174271 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Sep 09 23:03:18.175554 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.175537 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dtbbq"] Sep 09 23:03:18.175674 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.175610 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Sep 09 23:03:18.180210 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.180031 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Sep 09 23:03:18.181001 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.180959 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-xht2x"] Sep 09 23:03:18.191808 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.191786 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-4p2j4"] Sep 09 23:03:18.235357 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.235325 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4ebd018d-e004-4bd2-94ed-594b76e747ca-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-dtbbq\" (UID: \"4ebd018d-e004-4bd2-94ed-594b76e747ca\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dtbbq" Sep 09 23:03:18.235357 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.235357 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nn5k7\" (UniqueName: \"kubernetes.io/projected/476b7129-90d2-4523-97b4-43141066db1c-kube-api-access-nn5k7\") pod \"cluster-samples-operator-7cdcfcf76c-4p2j4\" (UID: \"476b7129-90d2-4523-97b4-43141066db1c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-4p2j4" Sep 09 23:03:18.235541 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.235399 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4ebd018d-e004-4bd2-94ed-594b76e747ca-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-dtbbq\" (UID: \"4ebd018d-e004-4bd2-94ed-594b76e747ca\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dtbbq" Sep 09 23:03:18.235541 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.235451 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnbfm\" (UniqueName: \"kubernetes.io/projected/4ebd018d-e004-4bd2-94ed-594b76e747ca-kube-api-access-xnbfm\") pod \"kube-storage-version-migrator-operator-8d7dd4675-dtbbq\" (UID: \"4ebd018d-e004-4bd2-94ed-594b76e747ca\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dtbbq" Sep 09 23:03:18.235541 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.235473 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/00cae89a-a6b7-401f-8fde-93e5f33890da-config\") pod \"service-ca-operator-859f94c4b-qctkt\" (UID: \"00cae89a-a6b7-401f-8fde-93e5f33890da\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qctkt" Sep 09 23:03:18.235541 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.235488 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3b6dd849-6a7e-4141-a460-e9ee0e7ff07c-config\") pod \"console-operator-85b454874b-xht2x\" (UID: \"3b6dd849-6a7e-4141-a460-e9ee0e7ff07c\") " pod="openshift-console-operator/console-operator-85b454874b-xht2x" Sep 09 23:03:18.235541 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.235511 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dt2p\" (UniqueName: \"kubernetes.io/projected/3b6dd849-6a7e-4141-a460-e9ee0e7ff07c-kube-api-access-9dt2p\") pod \"console-operator-85b454874b-xht2x\" (UID: \"3b6dd849-6a7e-4141-a460-e9ee0e7ff07c\") " pod="openshift-console-operator/console-operator-85b454874b-xht2x" Sep 09 23:03:18.235707 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.235546 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/476b7129-90d2-4523-97b4-43141066db1c-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-4p2j4\" (UID: \"476b7129-90d2-4523-97b4-43141066db1c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-4p2j4" Sep 09 23:03:18.235707 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.235563 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bv25n\" (UniqueName: \"kubernetes.io/projected/00cae89a-a6b7-401f-8fde-93e5f33890da-kube-api-access-bv25n\") pod \"service-ca-operator-859f94c4b-qctkt\" (UID: \"00cae89a-a6b7-401f-8fde-93e5f33890da\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qctkt" Sep 09 23:03:18.235707 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.235578 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3b6dd849-6a7e-4141-a460-e9ee0e7ff07c-trusted-ca\") pod \"console-operator-85b454874b-xht2x\" (UID: \"3b6dd849-6a7e-4141-a460-e9ee0e7ff07c\") " pod="openshift-console-operator/console-operator-85b454874b-xht2x" Sep 09 23:03:18.235707 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.235597 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3b6dd849-6a7e-4141-a460-e9ee0e7ff07c-serving-cert\") pod \"console-operator-85b454874b-xht2x\" (UID: \"3b6dd849-6a7e-4141-a460-e9ee0e7ff07c\") " pod="openshift-console-operator/console-operator-85b454874b-xht2x" Sep 09 23:03:18.235707 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.235629 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/00cae89a-a6b7-401f-8fde-93e5f33890da-serving-cert\") pod \"service-ca-operator-859f94c4b-qctkt\" (UID: \"00cae89a-a6b7-401f-8fde-93e5f33890da\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qctkt" Sep 09 23:03:18.250953 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.250925 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-74cbccb686-wxl6p"] Sep 09 23:03:18.253744 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.253729 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-74cbccb686-wxl6p" Sep 09 23:03:18.256989 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.256953 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Sep 09 23:03:18.257269 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.257248 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 09 23:03:18.257358 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.257300 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Sep 09 23:03:18.257502 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.257487 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-sn648\"" Sep 09 23:03:18.257610 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.257594 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 09 23:03:18.257989 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.257707 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Sep 09 23:03:18.258268 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.258251 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Sep 09 23:03:18.271122 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.271095 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-74cbccb686-wxl6p"] Sep 09 23:03:18.335913 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.335869 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-service-ca-bundle\") pod \"router-default-74cbccb686-wxl6p\" (UID: \"b7f670bd-13e2-4556-9c0f-ce8317a5ebb5\") " pod="openshift-ingress/router-default-74cbccb686-wxl6p" Sep 09 23:03:18.336104 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.335935 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/476b7129-90d2-4523-97b4-43141066db1c-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-4p2j4\" (UID: \"476b7129-90d2-4523-97b4-43141066db1c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-4p2j4" Sep 09 23:03:18.336104 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.336001 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3b6dd849-6a7e-4141-a460-e9ee0e7ff07c-trusted-ca\") pod \"console-operator-85b454874b-xht2x\" (UID: \"3b6dd849-6a7e-4141-a460-e9ee0e7ff07c\") " pod="openshift-console-operator/console-operator-85b454874b-xht2x" Sep 09 23:03:18.336104 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.336026 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4ebd018d-e004-4bd2-94ed-594b76e747ca-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-dtbbq\" (UID: \"4ebd018d-e004-4bd2-94ed-594b76e747ca\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dtbbq" Sep 09 23:03:18.336104 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.336069 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwl6b\" (UniqueName: \"kubernetes.io/projected/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-kube-api-access-bwl6b\") pod \"router-default-74cbccb686-wxl6p\" (UID: \"b7f670bd-13e2-4556-9c0f-ce8317a5ebb5\") " pod="openshift-ingress/router-default-74cbccb686-wxl6p" Sep 09 23:03:18.336104 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:18.336084 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 09 23:03:18.336289 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.336108 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dt2p\" (UniqueName: \"kubernetes.io/projected/3b6dd849-6a7e-4141-a460-e9ee0e7ff07c-kube-api-access-9dt2p\") pod \"console-operator-85b454874b-xht2x\" (UID: \"3b6dd849-6a7e-4141-a460-e9ee0e7ff07c\") " pod="openshift-console-operator/console-operator-85b454874b-xht2x" Sep 09 23:03:18.336289 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:18.336165 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/476b7129-90d2-4523-97b4-43141066db1c-samples-operator-tls podName:476b7129-90d2-4523-97b4-43141066db1c nodeName:}" failed. No retries permitted until 2026-09-09 23:03:18.836136243 +0000 UTC m=+260.186003266 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/476b7129-90d2-4523-97b4-43141066db1c-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-4p2j4" (UID: "476b7129-90d2-4523-97b4-43141066db1c") : secret "samples-operator-tls" not found Sep 09 23:03:18.336289 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.336249 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-metrics-certs\") pod \"router-default-74cbccb686-wxl6p\" (UID: \"b7f670bd-13e2-4556-9c0f-ce8317a5ebb5\") " pod="openshift-ingress/router-default-74cbccb686-wxl6p" Sep 09 23:03:18.336414 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.336293 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-stats-auth\") pod \"router-default-74cbccb686-wxl6p\" (UID: \"b7f670bd-13e2-4556-9c0f-ce8317a5ebb5\") " pod="openshift-ingress/router-default-74cbccb686-wxl6p" Sep 09 23:03:18.336414 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.336376 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nn5k7\" (UniqueName: \"kubernetes.io/projected/476b7129-90d2-4523-97b4-43141066db1c-kube-api-access-nn5k7\") pod \"cluster-samples-operator-7cdcfcf76c-4p2j4\" (UID: \"476b7129-90d2-4523-97b4-43141066db1c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-4p2j4" Sep 09 23:03:18.336493 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.336413 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bv25n\" (UniqueName: \"kubernetes.io/projected/00cae89a-a6b7-401f-8fde-93e5f33890da-kube-api-access-bv25n\") pod \"service-ca-operator-859f94c4b-qctkt\" (UID: \"00cae89a-a6b7-401f-8fde-93e5f33890da\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qctkt" Sep 09 23:03:18.336493 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.336454 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/00cae89a-a6b7-401f-8fde-93e5f33890da-config\") pod \"service-ca-operator-859f94c4b-qctkt\" (UID: \"00cae89a-a6b7-401f-8fde-93e5f33890da\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qctkt" Sep 09 23:03:18.336585 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.336566 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnbfm\" (UniqueName: \"kubernetes.io/projected/4ebd018d-e004-4bd2-94ed-594b76e747ca-kube-api-access-xnbfm\") pod \"kube-storage-version-migrator-operator-8d7dd4675-dtbbq\" (UID: \"4ebd018d-e004-4bd2-94ed-594b76e747ca\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dtbbq" Sep 09 23:03:18.336642 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.336602 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3b6dd849-6a7e-4141-a460-e9ee0e7ff07c-config\") pod \"console-operator-85b454874b-xht2x\" (UID: \"3b6dd849-6a7e-4141-a460-e9ee0e7ff07c\") " pod="openshift-console-operator/console-operator-85b454874b-xht2x" Sep 09 23:03:18.336695 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.336637 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-default-certificate\") pod \"router-default-74cbccb686-wxl6p\" (UID: \"b7f670bd-13e2-4556-9c0f-ce8317a5ebb5\") " pod="openshift-ingress/router-default-74cbccb686-wxl6p" Sep 09 23:03:18.336695 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.336666 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3b6dd849-6a7e-4141-a460-e9ee0e7ff07c-serving-cert\") pod \"console-operator-85b454874b-xht2x\" (UID: \"3b6dd849-6a7e-4141-a460-e9ee0e7ff07c\") " pod="openshift-console-operator/console-operator-85b454874b-xht2x" Sep 09 23:03:18.336799 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.336748 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4ebd018d-e004-4bd2-94ed-594b76e747ca-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-dtbbq\" (UID: \"4ebd018d-e004-4bd2-94ed-594b76e747ca\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dtbbq" Sep 09 23:03:18.336799 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.336787 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/00cae89a-a6b7-401f-8fde-93e5f33890da-serving-cert\") pod \"service-ca-operator-859f94c4b-qctkt\" (UID: \"00cae89a-a6b7-401f-8fde-93e5f33890da\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qctkt" Sep 09 23:03:18.336888 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.336856 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4ebd018d-e004-4bd2-94ed-594b76e747ca-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-dtbbq\" (UID: \"4ebd018d-e004-4bd2-94ed-594b76e747ca\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dtbbq" Sep 09 23:03:18.337107 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.337086 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3b6dd849-6a7e-4141-a460-e9ee0e7ff07c-trusted-ca\") pod \"console-operator-85b454874b-xht2x\" (UID: \"3b6dd849-6a7e-4141-a460-e9ee0e7ff07c\") " pod="openshift-console-operator/console-operator-85b454874b-xht2x" Sep 09 23:03:18.337107 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.337097 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/00cae89a-a6b7-401f-8fde-93e5f33890da-config\") pod \"service-ca-operator-859f94c4b-qctkt\" (UID: \"00cae89a-a6b7-401f-8fde-93e5f33890da\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qctkt" Sep 09 23:03:18.337261 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.337246 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3b6dd849-6a7e-4141-a460-e9ee0e7ff07c-config\") pod \"console-operator-85b454874b-xht2x\" (UID: \"3b6dd849-6a7e-4141-a460-e9ee0e7ff07c\") " pod="openshift-console-operator/console-operator-85b454874b-xht2x" Sep 09 23:03:18.338945 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.338915 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4ebd018d-e004-4bd2-94ed-594b76e747ca-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-dtbbq\" (UID: \"4ebd018d-e004-4bd2-94ed-594b76e747ca\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dtbbq" Sep 09 23:03:18.339091 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.339001 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/00cae89a-a6b7-401f-8fde-93e5f33890da-serving-cert\") pod \"service-ca-operator-859f94c4b-qctkt\" (UID: \"00cae89a-a6b7-401f-8fde-93e5f33890da\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qctkt" Sep 09 23:03:18.339153 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.339102 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3b6dd849-6a7e-4141-a460-e9ee0e7ff07c-serving-cert\") pod \"console-operator-85b454874b-xht2x\" (UID: \"3b6dd849-6a7e-4141-a460-e9ee0e7ff07c\") " pod="openshift-console-operator/console-operator-85b454874b-xht2x" Sep 09 23:03:18.348440 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.348416 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nn5k7\" (UniqueName: \"kubernetes.io/projected/476b7129-90d2-4523-97b4-43141066db1c-kube-api-access-nn5k7\") pod \"cluster-samples-operator-7cdcfcf76c-4p2j4\" (UID: \"476b7129-90d2-4523-97b4-43141066db1c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-4p2j4" Sep 09 23:03:18.349547 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.349516 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnbfm\" (UniqueName: \"kubernetes.io/projected/4ebd018d-e004-4bd2-94ed-594b76e747ca-kube-api-access-xnbfm\") pod \"kube-storage-version-migrator-operator-8d7dd4675-dtbbq\" (UID: \"4ebd018d-e004-4bd2-94ed-594b76e747ca\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dtbbq" Sep 09 23:03:18.349849 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.349832 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dt2p\" (UniqueName: \"kubernetes.io/projected/3b6dd849-6a7e-4141-a460-e9ee0e7ff07c-kube-api-access-9dt2p\") pod \"console-operator-85b454874b-xht2x\" (UID: \"3b6dd849-6a7e-4141-a460-e9ee0e7ff07c\") " pod="openshift-console-operator/console-operator-85b454874b-xht2x" Sep 09 23:03:18.350261 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.350246 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bv25n\" (UniqueName: \"kubernetes.io/projected/00cae89a-a6b7-401f-8fde-93e5f33890da-kube-api-access-bv25n\") pod \"service-ca-operator-859f94c4b-qctkt\" (UID: \"00cae89a-a6b7-401f-8fde-93e5f33890da\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qctkt" Sep 09 23:03:18.437647 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.437554 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-default-certificate\") pod \"router-default-74cbccb686-wxl6p\" (UID: \"b7f670bd-13e2-4556-9c0f-ce8317a5ebb5\") " pod="openshift-ingress/router-default-74cbccb686-wxl6p" Sep 09 23:03:18.437647 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.437611 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-service-ca-bundle\") pod \"router-default-74cbccb686-wxl6p\" (UID: \"b7f670bd-13e2-4556-9c0f-ce8317a5ebb5\") " pod="openshift-ingress/router-default-74cbccb686-wxl6p" Sep 09 23:03:18.437819 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.437754 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwl6b\" (UniqueName: \"kubernetes.io/projected/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-kube-api-access-bwl6b\") pod \"router-default-74cbccb686-wxl6p\" (UID: \"b7f670bd-13e2-4556-9c0f-ce8317a5ebb5\") " pod="openshift-ingress/router-default-74cbccb686-wxl6p" Sep 09 23:03:18.437819 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.437792 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-metrics-certs\") pod \"router-default-74cbccb686-wxl6p\" (UID: \"b7f670bd-13e2-4556-9c0f-ce8317a5ebb5\") " pod="openshift-ingress/router-default-74cbccb686-wxl6p" Sep 09 23:03:18.437819 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.437815 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-stats-auth\") pod \"router-default-74cbccb686-wxl6p\" (UID: \"b7f670bd-13e2-4556-9c0f-ce8317a5ebb5\") " pod="openshift-ingress/router-default-74cbccb686-wxl6p" Sep 09 23:03:18.437944 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:18.437853 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-service-ca-bundle podName:b7f670bd-13e2-4556-9c0f-ce8317a5ebb5 nodeName:}" failed. No retries permitted until 2026-09-09 23:03:18.937838391 +0000 UTC m=+260.287705404 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-service-ca-bundle") pod "router-default-74cbccb686-wxl6p" (UID: "b7f670bd-13e2-4556-9c0f-ce8317a5ebb5") : configmap references non-existent config key: service-ca.crt Sep 09 23:03:18.437944 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:18.437926 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 09 23:03:18.438116 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:18.438029 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-metrics-certs podName:b7f670bd-13e2-4556-9c0f-ce8317a5ebb5 nodeName:}" failed. No retries permitted until 2026-09-09 23:03:18.937992588 +0000 UTC m=+260.287859614 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-metrics-certs") pod "router-default-74cbccb686-wxl6p" (UID: "b7f670bd-13e2-4556-9c0f-ce8317a5ebb5") : secret "router-metrics-certs-default" not found Sep 09 23:03:18.440090 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.440069 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-default-certificate\") pod \"router-default-74cbccb686-wxl6p\" (UID: \"b7f670bd-13e2-4556-9c0f-ce8317a5ebb5\") " pod="openshift-ingress/router-default-74cbccb686-wxl6p" Sep 09 23:03:18.440191 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.440112 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-stats-auth\") pod \"router-default-74cbccb686-wxl6p\" (UID: \"b7f670bd-13e2-4556-9c0f-ce8317a5ebb5\") " pod="openshift-ingress/router-default-74cbccb686-wxl6p" Sep 09 23:03:18.446946 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.446923 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwl6b\" (UniqueName: \"kubernetes.io/projected/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-kube-api-access-bwl6b\") pod \"router-default-74cbccb686-wxl6p\" (UID: \"b7f670bd-13e2-4556-9c0f-ce8317a5ebb5\") " pod="openshift-ingress/router-default-74cbccb686-wxl6p" Sep 09 23:03:18.470357 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.470328 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qctkt" Sep 09 23:03:18.477382 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.477365 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dtbbq" Sep 09 23:03:18.490494 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.490469 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-xht2x" Sep 09 23:03:18.742233 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.742209 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-qctkt"] Sep 09 23:03:18.744488 ip-10-0-131-92 kubenswrapper[2580]: W0909 23:03:18.744463 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00cae89a_a6b7_401f_8fde_93e5f33890da.slice/crio-8699bd32cdd90fe9eadd8296ea689d0ed26cc4a752ea006c3ce393d7c2bb4a35 WatchSource:0}: Error finding container 8699bd32cdd90fe9eadd8296ea689d0ed26cc4a752ea006c3ce393d7c2bb4a35: Status 404 returned error can't find the container with id 8699bd32cdd90fe9eadd8296ea689d0ed26cc4a752ea006c3ce393d7c2bb4a35 Sep 09 23:03:18.763739 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.763680 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dtbbq"] Sep 09 23:03:18.765967 ip-10-0-131-92 kubenswrapper[2580]: W0909 23:03:18.765944 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ebd018d_e004_4bd2_94ed_594b76e747ca.slice/crio-102bc40bc7023d4363868447fa8561cfd7999d9a8463bb0da3ab8388a4c08c83 WatchSource:0}: Error finding container 102bc40bc7023d4363868447fa8561cfd7999d9a8463bb0da3ab8388a4c08c83: Status 404 returned error can't find the container with id 102bc40bc7023d4363868447fa8561cfd7999d9a8463bb0da3ab8388a4c08c83 Sep 09 23:03:18.768344 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.768319 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-xht2x"] Sep 09 23:03:18.770834 ip-10-0-131-92 kubenswrapper[2580]: W0909 23:03:18.770814 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b6dd849_6a7e_4141_a460_e9ee0e7ff07c.slice/crio-de4872e40b9e8aa81710fdb29c9fcc17f97578b7052e8d62942fc7c2a9fac927 WatchSource:0}: Error finding container de4872e40b9e8aa81710fdb29c9fcc17f97578b7052e8d62942fc7c2a9fac927: Status 404 returned error can't find the container with id de4872e40b9e8aa81710fdb29c9fcc17f97578b7052e8d62942fc7c2a9fac927 Sep 09 23:03:18.841602 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.841567 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/476b7129-90d2-4523-97b4-43141066db1c-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-4p2j4\" (UID: \"476b7129-90d2-4523-97b4-43141066db1c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-4p2j4" Sep 09 23:03:18.841765 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:18.841715 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 09 23:03:18.841819 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:18.841778 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/476b7129-90d2-4523-97b4-43141066db1c-samples-operator-tls podName:476b7129-90d2-4523-97b4-43141066db1c nodeName:}" failed. No retries permitted until 2026-09-09 23:03:19.841761488 +0000 UTC m=+261.191628500 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/476b7129-90d2-4523-97b4-43141066db1c-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-4p2j4" (UID: "476b7129-90d2-4523-97b4-43141066db1c") : secret "samples-operator-tls" not found Sep 09 23:03:18.942241 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.942201 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-service-ca-bundle\") pod \"router-default-74cbccb686-wxl6p\" (UID: \"b7f670bd-13e2-4556-9c0f-ce8317a5ebb5\") " pod="openshift-ingress/router-default-74cbccb686-wxl6p" Sep 09 23:03:18.942423 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.942361 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-metrics-certs\") pod \"router-default-74cbccb686-wxl6p\" (UID: \"b7f670bd-13e2-4556-9c0f-ce8317a5ebb5\") " pod="openshift-ingress/router-default-74cbccb686-wxl6p" Sep 09 23:03:18.942423 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:18.942407 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-service-ca-bundle podName:b7f670bd-13e2-4556-9c0f-ce8317a5ebb5 nodeName:}" failed. No retries permitted until 2026-09-09 23:03:19.942383155 +0000 UTC m=+261.292250190 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-service-ca-bundle") pod "router-default-74cbccb686-wxl6p" (UID: "b7f670bd-13e2-4556-9c0f-ce8317a5ebb5") : configmap references non-existent config key: service-ca.crt Sep 09 23:03:18.942506 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:18.942436 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 09 23:03:18.942506 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:18.942480 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-metrics-certs podName:b7f670bd-13e2-4556-9c0f-ce8317a5ebb5 nodeName:}" failed. No retries permitted until 2026-09-09 23:03:19.942468608 +0000 UTC m=+261.292335620 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-metrics-certs") pod "router-default-74cbccb686-wxl6p" (UID: "b7f670bd-13e2-4556-9c0f-ce8317a5ebb5") : secret "router-metrics-certs-default" not found Sep 09 23:03:18.982926 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.982887 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-xht2x" event={"ID":"3b6dd849-6a7e-4141-a460-e9ee0e7ff07c","Type":"ContainerStarted","Data":"de4872e40b9e8aa81710fdb29c9fcc17f97578b7052e8d62942fc7c2a9fac927"} Sep 09 23:03:18.983903 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.983880 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dtbbq" event={"ID":"4ebd018d-e004-4bd2-94ed-594b76e747ca","Type":"ContainerStarted","Data":"102bc40bc7023d4363868447fa8561cfd7999d9a8463bb0da3ab8388a4c08c83"} Sep 09 23:03:18.984819 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:18.984803 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qctkt" event={"ID":"00cae89a-a6b7-401f-8fde-93e5f33890da","Type":"ContainerStarted","Data":"8699bd32cdd90fe9eadd8296ea689d0ed26cc4a752ea006c3ce393d7c2bb4a35"} Sep 09 23:03:19.853801 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:19.853667 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/476b7129-90d2-4523-97b4-43141066db1c-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-4p2j4\" (UID: \"476b7129-90d2-4523-97b4-43141066db1c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-4p2j4" Sep 09 23:03:19.854480 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:19.853860 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 09 23:03:19.854480 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:19.853930 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/476b7129-90d2-4523-97b4-43141066db1c-samples-operator-tls podName:476b7129-90d2-4523-97b4-43141066db1c nodeName:}" failed. No retries permitted until 2026-09-09 23:03:21.853909535 +0000 UTC m=+263.203776548 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/476b7129-90d2-4523-97b4-43141066db1c-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-4p2j4" (UID: "476b7129-90d2-4523-97b4-43141066db1c") : secret "samples-operator-tls" not found Sep 09 23:03:19.954301 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:19.954251 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-service-ca-bundle\") pod \"router-default-74cbccb686-wxl6p\" (UID: \"b7f670bd-13e2-4556-9c0f-ce8317a5ebb5\") " pod="openshift-ingress/router-default-74cbccb686-wxl6p" Sep 09 23:03:19.954487 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:19.954391 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-metrics-certs\") pod \"router-default-74cbccb686-wxl6p\" (UID: \"b7f670bd-13e2-4556-9c0f-ce8317a5ebb5\") " pod="openshift-ingress/router-default-74cbccb686-wxl6p" Sep 09 23:03:19.954487 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:19.954430 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-service-ca-bundle podName:b7f670bd-13e2-4556-9c0f-ce8317a5ebb5 nodeName:}" failed. No retries permitted until 2026-09-09 23:03:21.954406967 +0000 UTC m=+263.304273995 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-service-ca-bundle") pod "router-default-74cbccb686-wxl6p" (UID: "b7f670bd-13e2-4556-9c0f-ce8317a5ebb5") : configmap references non-existent config key: service-ca.crt Sep 09 23:03:19.954616 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:19.954488 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 09 23:03:19.954667 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:19.954546 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-metrics-certs podName:b7f670bd-13e2-4556-9c0f-ce8317a5ebb5 nodeName:}" failed. No retries permitted until 2026-09-09 23:03:21.954529803 +0000 UTC m=+263.304396818 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-metrics-certs") pod "router-default-74cbccb686-wxl6p" (UID: "b7f670bd-13e2-4556-9c0f-ce8317a5ebb5") : secret "router-metrics-certs-default" not found Sep 09 23:03:21.872514 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:21.872478 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/476b7129-90d2-4523-97b4-43141066db1c-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-4p2j4\" (UID: \"476b7129-90d2-4523-97b4-43141066db1c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-4p2j4" Sep 09 23:03:21.872823 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:21.872660 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 09 23:03:21.872823 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:21.872735 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/476b7129-90d2-4523-97b4-43141066db1c-samples-operator-tls podName:476b7129-90d2-4523-97b4-43141066db1c nodeName:}" failed. No retries permitted until 2026-09-09 23:03:25.872715757 +0000 UTC m=+267.222582781 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/476b7129-90d2-4523-97b4-43141066db1c-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-4p2j4" (UID: "476b7129-90d2-4523-97b4-43141066db1c") : secret "samples-operator-tls" not found Sep 09 23:03:21.973802 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:21.973767 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-service-ca-bundle\") pod \"router-default-74cbccb686-wxl6p\" (UID: \"b7f670bd-13e2-4556-9c0f-ce8317a5ebb5\") " pod="openshift-ingress/router-default-74cbccb686-wxl6p" Sep 09 23:03:21.974024 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:21.973844 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-metrics-certs\") pod \"router-default-74cbccb686-wxl6p\" (UID: \"b7f670bd-13e2-4556-9c0f-ce8317a5ebb5\") " pod="openshift-ingress/router-default-74cbccb686-wxl6p" Sep 09 23:03:21.974024 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:21.973923 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-service-ca-bundle podName:b7f670bd-13e2-4556-9c0f-ce8317a5ebb5 nodeName:}" failed. No retries permitted until 2026-09-09 23:03:25.973905528 +0000 UTC m=+267.323772540 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-service-ca-bundle") pod "router-default-74cbccb686-wxl6p" (UID: "b7f670bd-13e2-4556-9c0f-ce8317a5ebb5") : configmap references non-existent config key: service-ca.crt Sep 09 23:03:21.974024 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:21.973968 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 09 23:03:21.974223 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:21.974035 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-metrics-certs podName:b7f670bd-13e2-4556-9c0f-ce8317a5ebb5 nodeName:}" failed. No retries permitted until 2026-09-09 23:03:25.97402311 +0000 UTC m=+267.323890122 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-metrics-certs") pod "router-default-74cbccb686-wxl6p" (UID: "b7f670bd-13e2-4556-9c0f-ce8317a5ebb5") : secret "router-metrics-certs-default" not found Sep 09 23:03:21.995739 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:21.995707 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qctkt" event={"ID":"00cae89a-a6b7-401f-8fde-93e5f33890da","Type":"ContainerStarted","Data":"05751b8cad31a9822128f9421a8f098b769566ef65cf5b225c2947cae1365ccc"} Sep 09 23:03:21.997530 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:21.997500 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-xht2x" event={"ID":"3b6dd849-6a7e-4141-a460-e9ee0e7ff07c","Type":"ContainerStarted","Data":"7eb967edc13813d1749d940ec0ebc77ec6d3db76d9c8ddfa669d0a07ca6e1938"} Sep 09 23:03:21.997723 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:21.997697 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-xht2x" Sep 09 23:03:21.999363 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:21.999334 2580 patch_prober.go:28] interesting pod/console-operator-85b454874b-xht2x container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.12:8443/readyz\": dial tcp 10.132.0.12:8443: connect: connection refused" start-of-body= Sep 09 23:03:21.999484 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:21.999396 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-85b454874b-xht2x" podUID="3b6dd849-6a7e-4141-a460-e9ee0e7ff07c" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.12:8443/readyz\": dial tcp 10.132.0.12:8443: connect: connection refused" Sep 09 23:03:21.999768 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:21.999745 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dtbbq" event={"ID":"4ebd018d-e004-4bd2-94ed-594b76e747ca","Type":"ContainerStarted","Data":"931ab674255655e97aa7d917eee3c9587333ee3f1073c7fa449f3648ee2031cf"} Sep 09 23:03:22.007134 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:22.007113 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-86mkf"] Sep 09 23:03:22.010029 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:22.010002 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-86mkf" Sep 09 23:03:22.012567 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:22.012549 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 09 23:03:22.012718 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:22.012632 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 09 23:03:22.012718 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:22.012641 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 09 23:03:22.012945 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:22.012903 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 09 23:03:22.013068 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:22.013051 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nh9jh\"" Sep 09 23:03:22.018907 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:22.018852 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qctkt" podStartSLOduration=0.904721732 podStartE2EDuration="4.018833806s" totalImagesPullingTime="3.114112074s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:03:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:03:22.017700639 +0000 UTC m=+263.367567676" watchObservedRunningTime="2026-09-09 23:03:22.018833806 +0000 UTC m=+263.368700842" Sep 09 23:03:22.020925 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:22.020905 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-86mkf"] Sep 09 23:03:22.036445 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:22.036398 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-xht2x" podStartSLOduration=0.95613594 podStartE2EDuration="4.036382222s" totalImagesPullingTime="3.080246282s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:03:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:03:22.035712985 +0000 UTC m=+263.385580031" watchObservedRunningTime="2026-09-09 23:03:22.036382222 +0000 UTC m=+263.386249256" Sep 09 23:03:22.050717 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:22.050614 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dtbbq" podStartSLOduration=0.962420977 podStartE2EDuration="4.050595082s" totalImagesPullingTime="3.088174105s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:03:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:03:22.049669472 +0000 UTC m=+263.399536499" watchObservedRunningTime="2026-09-09 23:03:22.050595082 +0000 UTC m=+263.400462117" Sep 09 23:03:22.074248 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:22.074213 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7e149a18-7de7-43d3-89ca-14e842fa6d77-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-86mkf\" (UID: \"7e149a18-7de7-43d3-89ca-14e842fa6d77\") " pod="openshift-insights/insights-runtime-extractor-86mkf" Sep 09 23:03:22.074417 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:22.074259 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7e149a18-7de7-43d3-89ca-14e842fa6d77-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-86mkf\" (UID: \"7e149a18-7de7-43d3-89ca-14e842fa6d77\") " pod="openshift-insights/insights-runtime-extractor-86mkf" Sep 09 23:03:22.074479 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:22.074440 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e149a18-7de7-43d3-89ca-14e842fa6d77-data-volume\") pod \"insights-runtime-extractor-86mkf\" (UID: \"7e149a18-7de7-43d3-89ca-14e842fa6d77\") " pod="openshift-insights/insights-runtime-extractor-86mkf" Sep 09 23:03:22.074627 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:22.074603 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-296zr\" (UniqueName: \"kubernetes.io/projected/7e149a18-7de7-43d3-89ca-14e842fa6d77-kube-api-access-296zr\") pod \"insights-runtime-extractor-86mkf\" (UID: \"7e149a18-7de7-43d3-89ca-14e842fa6d77\") " pod="openshift-insights/insights-runtime-extractor-86mkf" Sep 09 23:03:22.074841 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:22.074801 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7e149a18-7de7-43d3-89ca-14e842fa6d77-crio-socket\") pod \"insights-runtime-extractor-86mkf\" (UID: \"7e149a18-7de7-43d3-89ca-14e842fa6d77\") " pod="openshift-insights/insights-runtime-extractor-86mkf" Sep 09 23:03:22.176253 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:22.176217 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7e149a18-7de7-43d3-89ca-14e842fa6d77-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-86mkf\" (UID: \"7e149a18-7de7-43d3-89ca-14e842fa6d77\") " pod="openshift-insights/insights-runtime-extractor-86mkf" Sep 09 23:03:22.176253 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:22.176258 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7e149a18-7de7-43d3-89ca-14e842fa6d77-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-86mkf\" (UID: \"7e149a18-7de7-43d3-89ca-14e842fa6d77\") " pod="openshift-insights/insights-runtime-extractor-86mkf" Sep 09 23:03:22.176509 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:22.176295 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e149a18-7de7-43d3-89ca-14e842fa6d77-data-volume\") pod \"insights-runtime-extractor-86mkf\" (UID: \"7e149a18-7de7-43d3-89ca-14e842fa6d77\") " pod="openshift-insights/insights-runtime-extractor-86mkf" Sep 09 23:03:22.176509 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:22.176364 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-296zr\" (UniqueName: \"kubernetes.io/projected/7e149a18-7de7-43d3-89ca-14e842fa6d77-kube-api-access-296zr\") pod \"insights-runtime-extractor-86mkf\" (UID: \"7e149a18-7de7-43d3-89ca-14e842fa6d77\") " pod="openshift-insights/insights-runtime-extractor-86mkf" Sep 09 23:03:22.176509 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:22.176389 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 23:03:22.176509 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:22.176411 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7e149a18-7de7-43d3-89ca-14e842fa6d77-crio-socket\") pod \"insights-runtime-extractor-86mkf\" (UID: \"7e149a18-7de7-43d3-89ca-14e842fa6d77\") " pod="openshift-insights/insights-runtime-extractor-86mkf" Sep 09 23:03:22.176509 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:22.176422 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 23:03:22.176509 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:22.176456 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e149a18-7de7-43d3-89ca-14e842fa6d77-insights-runtime-extractor-tls podName:7e149a18-7de7-43d3-89ca-14e842fa6d77 nodeName:}" failed. No retries permitted until 2026-09-09 23:03:22.676436362 +0000 UTC m=+264.026303389 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7e149a18-7de7-43d3-89ca-14e842fa6d77-insights-runtime-extractor-tls") pod "insights-runtime-extractor-86mkf" (UID: "7e149a18-7de7-43d3-89ca-14e842fa6d77") : secret "insights-runtime-extractor-tls" not found Sep 09 23:03:22.176509 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:22.176487 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7e149a18-7de7-43d3-89ca-14e842fa6d77-kube-rbac-proxy-cm podName:7e149a18-7de7-43d3-89ca-14e842fa6d77 nodeName:}" failed. No retries permitted until 2026-09-09 23:03:22.676469691 +0000 UTC m=+264.026336713 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7e149a18-7de7-43d3-89ca-14e842fa6d77-kube-rbac-proxy-cm") pod "insights-runtime-extractor-86mkf" (UID: "7e149a18-7de7-43d3-89ca-14e842fa6d77") : configmap "kube-rbac-proxy" not found Sep 09 23:03:22.176509 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:22.176499 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7e149a18-7de7-43d3-89ca-14e842fa6d77-crio-socket\") pod \"insights-runtime-extractor-86mkf\" (UID: \"7e149a18-7de7-43d3-89ca-14e842fa6d77\") " pod="openshift-insights/insights-runtime-extractor-86mkf" Sep 09 23:03:22.176941 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:22.176686 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e149a18-7de7-43d3-89ca-14e842fa6d77-data-volume\") pod \"insights-runtime-extractor-86mkf\" (UID: \"7e149a18-7de7-43d3-89ca-14e842fa6d77\") " pod="openshift-insights/insights-runtime-extractor-86mkf" Sep 09 23:03:22.185007 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:22.184964 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-296zr\" (UniqueName: \"kubernetes.io/projected/7e149a18-7de7-43d3-89ca-14e842fa6d77-kube-api-access-296zr\") pod \"insights-runtime-extractor-86mkf\" (UID: \"7e149a18-7de7-43d3-89ca-14e842fa6d77\") " pod="openshift-insights/insights-runtime-extractor-86mkf" Sep 09 23:03:22.680632 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:22.680587 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7e149a18-7de7-43d3-89ca-14e842fa6d77-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-86mkf\" (UID: \"7e149a18-7de7-43d3-89ca-14e842fa6d77\") " pod="openshift-insights/insights-runtime-extractor-86mkf" Sep 09 23:03:22.680833 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:22.680720 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7e149a18-7de7-43d3-89ca-14e842fa6d77-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-86mkf\" (UID: \"7e149a18-7de7-43d3-89ca-14e842fa6d77\") " pod="openshift-insights/insights-runtime-extractor-86mkf" Sep 09 23:03:22.680833 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:22.680724 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 23:03:22.680833 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:22.680791 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7e149a18-7de7-43d3-89ca-14e842fa6d77-kube-rbac-proxy-cm podName:7e149a18-7de7-43d3-89ca-14e842fa6d77 nodeName:}" failed. No retries permitted until 2026-09-09 23:03:23.680772333 +0000 UTC m=+265.030639358 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7e149a18-7de7-43d3-89ca-14e842fa6d77-kube-rbac-proxy-cm") pod "insights-runtime-extractor-86mkf" (UID: "7e149a18-7de7-43d3-89ca-14e842fa6d77") : configmap "kube-rbac-proxy" not found Sep 09 23:03:22.680833 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:22.680803 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 23:03:22.681073 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:22.680841 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e149a18-7de7-43d3-89ca-14e842fa6d77-insights-runtime-extractor-tls podName:7e149a18-7de7-43d3-89ca-14e842fa6d77 nodeName:}" failed. No retries permitted until 2026-09-09 23:03:23.680830499 +0000 UTC m=+265.030697511 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7e149a18-7de7-43d3-89ca-14e842fa6d77-insights-runtime-extractor-tls") pod "insights-runtime-extractor-86mkf" (UID: "7e149a18-7de7-43d3-89ca-14e842fa6d77") : secret "insights-runtime-extractor-tls" not found Sep 09 23:03:23.004448 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:23.004368 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xht2x_3b6dd849-6a7e-4141-a460-e9ee0e7ff07c/console-operator/0.log" Sep 09 23:03:23.004448 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:23.004418 2580 generic.go:356] "Generic (PLEG): container finished" podID="3b6dd849-6a7e-4141-a460-e9ee0e7ff07c" containerID="7eb967edc13813d1749d940ec0ebc77ec6d3db76d9c8ddfa669d0a07ca6e1938" exitCode=255 Sep 09 23:03:23.004842 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:23.004502 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-xht2x" event={"ID":"3b6dd849-6a7e-4141-a460-e9ee0e7ff07c","Type":"ContainerDied","Data":"7eb967edc13813d1749d940ec0ebc77ec6d3db76d9c8ddfa669d0a07ca6e1938"} Sep 09 23:03:23.004842 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:23.004786 2580 scope.go:117] "RemoveContainer" containerID="7eb967edc13813d1749d940ec0ebc77ec6d3db76d9c8ddfa669d0a07ca6e1938" Sep 09 23:03:23.689803 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:23.689762 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7e149a18-7de7-43d3-89ca-14e842fa6d77-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-86mkf\" (UID: \"7e149a18-7de7-43d3-89ca-14e842fa6d77\") " pod="openshift-insights/insights-runtime-extractor-86mkf" Sep 09 23:03:23.689803 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:23.689804 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7e149a18-7de7-43d3-89ca-14e842fa6d77-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-86mkf\" (UID: \"7e149a18-7de7-43d3-89ca-14e842fa6d77\") " pod="openshift-insights/insights-runtime-extractor-86mkf" Sep 09 23:03:23.690067 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:23.689898 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 23:03:23.690067 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:23.689915 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 23:03:23.690067 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:23.689957 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7e149a18-7de7-43d3-89ca-14e842fa6d77-kube-rbac-proxy-cm podName:7e149a18-7de7-43d3-89ca-14e842fa6d77 nodeName:}" failed. No retries permitted until 2026-09-09 23:03:25.689942963 +0000 UTC m=+267.039809977 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7e149a18-7de7-43d3-89ca-14e842fa6d77-kube-rbac-proxy-cm") pod "insights-runtime-extractor-86mkf" (UID: "7e149a18-7de7-43d3-89ca-14e842fa6d77") : configmap "kube-rbac-proxy" not found Sep 09 23:03:23.690067 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:23.689990 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e149a18-7de7-43d3-89ca-14e842fa6d77-insights-runtime-extractor-tls podName:7e149a18-7de7-43d3-89ca-14e842fa6d77 nodeName:}" failed. No retries permitted until 2026-09-09 23:03:25.689966143 +0000 UTC m=+267.039833155 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7e149a18-7de7-43d3-89ca-14e842fa6d77-insights-runtime-extractor-tls") pod "insights-runtime-extractor-86mkf" (UID: "7e149a18-7de7-43d3-89ca-14e842fa6d77") : secret "insights-runtime-extractor-tls" not found Sep 09 23:03:24.009310 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:24.009237 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xht2x_3b6dd849-6a7e-4141-a460-e9ee0e7ff07c/console-operator/1.log" Sep 09 23:03:24.009714 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:24.009696 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xht2x_3b6dd849-6a7e-4141-a460-e9ee0e7ff07c/console-operator/0.log" Sep 09 23:03:24.009756 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:24.009734 2580 generic.go:356] "Generic (PLEG): container finished" podID="3b6dd849-6a7e-4141-a460-e9ee0e7ff07c" containerID="058b36e12be4939a58c04a7bca6e0f7b37f347f9b66c7bd51a7f31aa3069e777" exitCode=255 Sep 09 23:03:24.009789 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:24.009766 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-xht2x" event={"ID":"3b6dd849-6a7e-4141-a460-e9ee0e7ff07c","Type":"ContainerDied","Data":"058b36e12be4939a58c04a7bca6e0f7b37f347f9b66c7bd51a7f31aa3069e777"} Sep 09 23:03:24.009829 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:24.009818 2580 scope.go:117] "RemoveContainer" containerID="7eb967edc13813d1749d940ec0ebc77ec6d3db76d9c8ddfa669d0a07ca6e1938" Sep 09 23:03:24.010084 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:24.010068 2580 scope.go:117] "RemoveContainer" containerID="058b36e12be4939a58c04a7bca6e0f7b37f347f9b66c7bd51a7f31aa3069e777" Sep 09 23:03:24.010265 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:24.010238 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-xht2x_openshift-console-operator(3b6dd849-6a7e-4141-a460-e9ee0e7ff07c)\"" pod="openshift-console-operator/console-operator-85b454874b-xht2x" podUID="3b6dd849-6a7e-4141-a460-e9ee0e7ff07c" Sep 09 23:03:24.154992 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:24.154949 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lwqr2_dadd077d-e65f-49c0-a1bf-b8299ef2bb89/dns-node-resolver/0.log" Sep 09 23:03:24.954984 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:24.954942 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jx58q_0fe9b158-c1b2-498d-aecc-d32aee3c51c6/node-ca/0.log" Sep 09 23:03:25.013880 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:25.013854 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xht2x_3b6dd849-6a7e-4141-a460-e9ee0e7ff07c/console-operator/1.log" Sep 09 23:03:25.014277 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:25.014203 2580 scope.go:117] "RemoveContainer" containerID="058b36e12be4939a58c04a7bca6e0f7b37f347f9b66c7bd51a7f31aa3069e777" Sep 09 23:03:25.014380 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:25.014360 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-xht2x_openshift-console-operator(3b6dd849-6a7e-4141-a460-e9ee0e7ff07c)\"" pod="openshift-console-operator/console-operator-85b454874b-xht2x" podUID="3b6dd849-6a7e-4141-a460-e9ee0e7ff07c" Sep 09 23:03:25.554235 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:25.554203 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5dl4c_d1a033d1-6eb8-472d-95dc-5a1c26b60867/serve-healthcheck-canary/0.log" Sep 09 23:03:25.707176 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:25.707126 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7e149a18-7de7-43d3-89ca-14e842fa6d77-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-86mkf\" (UID: \"7e149a18-7de7-43d3-89ca-14e842fa6d77\") " pod="openshift-insights/insights-runtime-extractor-86mkf" Sep 09 23:03:25.707176 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:25.707179 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7e149a18-7de7-43d3-89ca-14e842fa6d77-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-86mkf\" (UID: \"7e149a18-7de7-43d3-89ca-14e842fa6d77\") " pod="openshift-insights/insights-runtime-extractor-86mkf" Sep 09 23:03:25.707430 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:25.707298 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 23:03:25.707430 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:25.707319 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 23:03:25.707430 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:25.707376 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e149a18-7de7-43d3-89ca-14e842fa6d77-insights-runtime-extractor-tls podName:7e149a18-7de7-43d3-89ca-14e842fa6d77 nodeName:}" failed. No retries permitted until 2026-09-09 23:03:29.707355889 +0000 UTC m=+271.057222904 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7e149a18-7de7-43d3-89ca-14e842fa6d77-insights-runtime-extractor-tls") pod "insights-runtime-extractor-86mkf" (UID: "7e149a18-7de7-43d3-89ca-14e842fa6d77") : secret "insights-runtime-extractor-tls" not found Sep 09 23:03:25.707430 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:25.707397 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7e149a18-7de7-43d3-89ca-14e842fa6d77-kube-rbac-proxy-cm podName:7e149a18-7de7-43d3-89ca-14e842fa6d77 nodeName:}" failed. No retries permitted until 2026-09-09 23:03:29.707385638 +0000 UTC m=+271.057252650 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7e149a18-7de7-43d3-89ca-14e842fa6d77-kube-rbac-proxy-cm") pod "insights-runtime-extractor-86mkf" (UID: "7e149a18-7de7-43d3-89ca-14e842fa6d77") : configmap "kube-rbac-proxy" not found Sep 09 23:03:25.909645 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:25.909603 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/476b7129-90d2-4523-97b4-43141066db1c-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-4p2j4\" (UID: \"476b7129-90d2-4523-97b4-43141066db1c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-4p2j4" Sep 09 23:03:25.909827 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:25.909748 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 09 23:03:25.909827 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:25.909812 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/476b7129-90d2-4523-97b4-43141066db1c-samples-operator-tls podName:476b7129-90d2-4523-97b4-43141066db1c nodeName:}" failed. No retries permitted until 2026-09-09 23:03:33.909797138 +0000 UTC m=+275.259664149 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/476b7129-90d2-4523-97b4-43141066db1c-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-4p2j4" (UID: "476b7129-90d2-4523-97b4-43141066db1c") : secret "samples-operator-tls" not found Sep 09 23:03:26.010354 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:26.010316 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-service-ca-bundle\") pod \"router-default-74cbccb686-wxl6p\" (UID: \"b7f670bd-13e2-4556-9c0f-ce8317a5ebb5\") " pod="openshift-ingress/router-default-74cbccb686-wxl6p" Sep 09 23:03:26.010510 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:26.010411 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-metrics-certs\") pod \"router-default-74cbccb686-wxl6p\" (UID: \"b7f670bd-13e2-4556-9c0f-ce8317a5ebb5\") " pod="openshift-ingress/router-default-74cbccb686-wxl6p" Sep 09 23:03:26.010510 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:26.010475 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-service-ca-bundle podName:b7f670bd-13e2-4556-9c0f-ce8317a5ebb5 nodeName:}" failed. No retries permitted until 2026-09-09 23:03:34.010457365 +0000 UTC m=+275.360324377 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-service-ca-bundle") pod "router-default-74cbccb686-wxl6p" (UID: "b7f670bd-13e2-4556-9c0f-ce8317a5ebb5") : configmap references non-existent config key: service-ca.crt Sep 09 23:03:26.010639 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:26.010535 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 09 23:03:26.010639 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:26.010590 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-metrics-certs podName:b7f670bd-13e2-4556-9c0f-ce8317a5ebb5 nodeName:}" failed. No retries permitted until 2026-09-09 23:03:34.010575272 +0000 UTC m=+275.360442300 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-metrics-certs") pod "router-default-74cbccb686-wxl6p" (UID: "b7f670bd-13e2-4556-9c0f-ce8317a5ebb5") : secret "router-metrics-certs-default" not found Sep 09 23:03:28.491354 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:28.491272 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-xht2x" Sep 09 23:03:28.491736 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:28.491667 2580 scope.go:117] "RemoveContainer" containerID="058b36e12be4939a58c04a7bca6e0f7b37f347f9b66c7bd51a7f31aa3069e777" Sep 09 23:03:28.491823 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:28.491804 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-xht2x_openshift-console-operator(3b6dd849-6a7e-4141-a460-e9ee0e7ff07c)\"" pod="openshift-console-operator/console-operator-85b454874b-xht2x" podUID="3b6dd849-6a7e-4141-a460-e9ee0e7ff07c" Sep 09 23:03:29.742212 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:29.742172 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7e149a18-7de7-43d3-89ca-14e842fa6d77-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-86mkf\" (UID: \"7e149a18-7de7-43d3-89ca-14e842fa6d77\") " pod="openshift-insights/insights-runtime-extractor-86mkf" Sep 09 23:03:29.742553 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:29.742223 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7e149a18-7de7-43d3-89ca-14e842fa6d77-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-86mkf\" (UID: \"7e149a18-7de7-43d3-89ca-14e842fa6d77\") " pod="openshift-insights/insights-runtime-extractor-86mkf" Sep 09 23:03:29.742553 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:29.742306 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 23:03:29.742553 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:29.742325 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 23:03:29.742553 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:29.742375 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e149a18-7de7-43d3-89ca-14e842fa6d77-insights-runtime-extractor-tls podName:7e149a18-7de7-43d3-89ca-14e842fa6d77 nodeName:}" failed. No retries permitted until 2026-09-09 23:03:37.742354601 +0000 UTC m=+279.092221622 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7e149a18-7de7-43d3-89ca-14e842fa6d77-insights-runtime-extractor-tls") pod "insights-runtime-extractor-86mkf" (UID: "7e149a18-7de7-43d3-89ca-14e842fa6d77") : secret "insights-runtime-extractor-tls" not found Sep 09 23:03:29.742553 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:29.742392 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7e149a18-7de7-43d3-89ca-14e842fa6d77-kube-rbac-proxy-cm podName:7e149a18-7de7-43d3-89ca-14e842fa6d77 nodeName:}" failed. No retries permitted until 2026-09-09 23:03:37.742384184 +0000 UTC m=+279.092251203 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7e149a18-7de7-43d3-89ca-14e842fa6d77-kube-rbac-proxy-cm") pod "insights-runtime-extractor-86mkf" (UID: "7e149a18-7de7-43d3-89ca-14e842fa6d77") : configmap "kube-rbac-proxy" not found Sep 09 23:03:31.997730 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:31.997693 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-xht2x" Sep 09 23:03:31.998132 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:31.998044 2580 scope.go:117] "RemoveContainer" containerID="058b36e12be4939a58c04a7bca6e0f7b37f347f9b66c7bd51a7f31aa3069e777" Sep 09 23:03:31.998208 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:31.998189 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-xht2x_openshift-console-operator(3b6dd849-6a7e-4141-a460-e9ee0e7ff07c)\"" pod="openshift-console-operator/console-operator-85b454874b-xht2x" podUID="3b6dd849-6a7e-4141-a460-e9ee0e7ff07c" Sep 09 23:03:33.975252 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:33.975214 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/476b7129-90d2-4523-97b4-43141066db1c-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-4p2j4\" (UID: \"476b7129-90d2-4523-97b4-43141066db1c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-4p2j4" Sep 09 23:03:33.977652 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:33.977627 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/476b7129-90d2-4523-97b4-43141066db1c-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-4p2j4\" (UID: \"476b7129-90d2-4523-97b4-43141066db1c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-4p2j4" Sep 09 23:03:34.076244 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:34.076210 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-metrics-certs\") pod \"router-default-74cbccb686-wxl6p\" (UID: \"b7f670bd-13e2-4556-9c0f-ce8317a5ebb5\") " pod="openshift-ingress/router-default-74cbccb686-wxl6p" Sep 09 23:03:34.076409 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:34.076326 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-service-ca-bundle\") pod \"router-default-74cbccb686-wxl6p\" (UID: \"b7f670bd-13e2-4556-9c0f-ce8317a5ebb5\") " pod="openshift-ingress/router-default-74cbccb686-wxl6p" Sep 09 23:03:34.076409 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:34.076350 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 09 23:03:34.076798 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:34.076769 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-metrics-certs podName:b7f670bd-13e2-4556-9c0f-ce8317a5ebb5 nodeName:}" failed. No retries permitted until 2026-09-09 23:03:50.076391284 +0000 UTC m=+291.426258297 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-metrics-certs") pod "router-default-74cbccb686-wxl6p" (UID: "b7f670bd-13e2-4556-9c0f-ce8317a5ebb5") : secret "router-metrics-certs-default" not found Sep 09 23:03:34.077563 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:34.077539 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-service-ca-bundle podName:b7f670bd-13e2-4556-9c0f-ce8317a5ebb5 nodeName:}" failed. No retries permitted until 2026-09-09 23:03:50.077515691 +0000 UTC m=+291.427382718 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-service-ca-bundle") pod "router-default-74cbccb686-wxl6p" (UID: "b7f670bd-13e2-4556-9c0f-ce8317a5ebb5") : configmap references non-existent config key: service-ca.crt Sep 09 23:03:34.084683 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:34.084654 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-4p2j4" Sep 09 23:03:34.359104 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:34.359027 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-4p2j4"] Sep 09 23:03:35.043801 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:35.043759 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-4p2j4" event={"ID":"476b7129-90d2-4523-97b4-43141066db1c","Type":"ContainerStarted","Data":"7176bf13c973b17eefec22c50df67ee46b31cfabd84d53e58ba8af10c405280a"} Sep 09 23:03:37.050909 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:37.050876 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-4p2j4" event={"ID":"476b7129-90d2-4523-97b4-43141066db1c","Type":"ContainerStarted","Data":"9061e648c80a967938a3945867c22c6a769df0b98901c273326d3d5ecb3b993b"} Sep 09 23:03:37.051302 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:37.050934 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-4p2j4" event={"ID":"476b7129-90d2-4523-97b4-43141066db1c","Type":"ContainerStarted","Data":"1ea3744d939cf88c3e19e2bf2533dcf90b78029c68df35de504efcea1b7b6c9c"} Sep 09 23:03:37.081344 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:37.081294 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-4p2j4" podStartSLOduration=17.406948548 podStartE2EDuration="19.081276843s" totalImagesPullingTime="1.674328295s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:03:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:03:37.081200107 +0000 UTC m=+278.431067141" watchObservedRunningTime="2026-09-09 23:03:37.081276843 +0000 UTC m=+278.431143876" Sep 09 23:03:37.806477 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:37.806427 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7e149a18-7de7-43d3-89ca-14e842fa6d77-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-86mkf\" (UID: \"7e149a18-7de7-43d3-89ca-14e842fa6d77\") " pod="openshift-insights/insights-runtime-extractor-86mkf" Sep 09 23:03:37.806477 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:37.806479 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7e149a18-7de7-43d3-89ca-14e842fa6d77-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-86mkf\" (UID: \"7e149a18-7de7-43d3-89ca-14e842fa6d77\") " pod="openshift-insights/insights-runtime-extractor-86mkf" Sep 09 23:03:37.806697 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:37.806582 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 23:03:37.806697 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:37.806651 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e149a18-7de7-43d3-89ca-14e842fa6d77-insights-runtime-extractor-tls podName:7e149a18-7de7-43d3-89ca-14e842fa6d77 nodeName:}" failed. No retries permitted until 2026-09-09 23:03:53.80663504 +0000 UTC m=+295.156502052 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7e149a18-7de7-43d3-89ca-14e842fa6d77-insights-runtime-extractor-tls") pod "insights-runtime-extractor-86mkf" (UID: "7e149a18-7de7-43d3-89ca-14e842fa6d77") : secret "insights-runtime-extractor-tls" not found Sep 09 23:03:37.807002 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:37.806962 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7e149a18-7de7-43d3-89ca-14e842fa6d77-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-86mkf\" (UID: \"7e149a18-7de7-43d3-89ca-14e842fa6d77\") " pod="openshift-insights/insights-runtime-extractor-86mkf" Sep 09 23:03:38.732203 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:38.732137 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-69855d874b-xvwj9" podUID="c15d7152-f7f5-4d4c-8d66-4f955b5b9725" Sep 09 23:03:38.732203 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:38.732153 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-b6zvm" podUID="6251fed2-5675-42f5-b0d1-788e1a894092" Sep 09 23:03:39.057436 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:39.057356 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69855d874b-xvwj9" Sep 09 23:03:39.057598 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:39.057357 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b6zvm" Sep 09 23:03:42.543483 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:42.543441 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-registry-tls\") pod \"image-registry-69855d874b-xvwj9\" (UID: \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\") " pod="openshift-image-registry/image-registry-69855d874b-xvwj9" Sep 09 23:03:42.543962 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:42.543523 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6251fed2-5675-42f5-b0d1-788e1a894092-metrics-tls\") pod \"dns-default-b6zvm\" (UID: \"6251fed2-5675-42f5-b0d1-788e1a894092\") " pod="openshift-dns/dns-default-b6zvm" Sep 09 23:03:42.543962 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:42.543569 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5f001595-27ff-43d7-94d2-7ddc18d99671-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-zmvn9\" (UID: \"5f001595-27ff-43d7-94d2-7ddc18d99671\") " pod="openshift-network-console/networking-console-plugin-79977f9844-zmvn9" Sep 09 23:03:42.545854 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:42.545830 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5f001595-27ff-43d7-94d2-7ddc18d99671-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-zmvn9\" (UID: \"5f001595-27ff-43d7-94d2-7ddc18d99671\") " pod="openshift-network-console/networking-console-plugin-79977f9844-zmvn9" Sep 09 23:03:42.546344 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:42.546320 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-registry-tls\") pod \"image-registry-69855d874b-xvwj9\" (UID: \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\") " pod="openshift-image-registry/image-registry-69855d874b-xvwj9" Sep 09 23:03:42.546439 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:42.546326 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6251fed2-5675-42f5-b0d1-788e1a894092-metrics-tls\") pod \"dns-default-b6zvm\" (UID: \"6251fed2-5675-42f5-b0d1-788e1a894092\") " pod="openshift-dns/dns-default-b6zvm" Sep 09 23:03:42.662028 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:42.661997 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-p5vxk\"" Sep 09 23:03:42.662028 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:42.662015 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-t7bbt\"" Sep 09 23:03:42.668479 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:42.668458 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69855d874b-xvwj9" Sep 09 23:03:42.668579 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:42.668489 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b6zvm" Sep 09 23:03:42.675718 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:42.675699 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-wjdkt\"" Sep 09 23:03:42.683928 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:42.683899 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-zmvn9" Sep 09 23:03:42.945939 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:42.945909 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-b6zvm"] Sep 09 23:03:42.948893 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:42.948869 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-69855d874b-xvwj9"] Sep 09 23:03:42.950332 ip-10-0-131-92 kubenswrapper[2580]: W0909 23:03:42.950306 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6251fed2_5675_42f5_b0d1_788e1a894092.slice/crio-322c16066645e0d9359acae20971730f2953133f8163a356bebfa7b05f868c10 WatchSource:0}: Error finding container 322c16066645e0d9359acae20971730f2953133f8163a356bebfa7b05f868c10: Status 404 returned error can't find the container with id 322c16066645e0d9359acae20971730f2953133f8163a356bebfa7b05f868c10 Sep 09 23:03:42.951111 ip-10-0-131-92 kubenswrapper[2580]: W0909 23:03:42.951087 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc15d7152_f7f5_4d4c_8d66_4f955b5b9725.slice/crio-217972ed9b37df73b542ceb7d6374950a0a40e87d9d1d0c2d4ed232751b70034 WatchSource:0}: Error finding container 217972ed9b37df73b542ceb7d6374950a0a40e87d9d1d0c2d4ed232751b70034: Status 404 returned error can't find the container with id 217972ed9b37df73b542ceb7d6374950a0a40e87d9d1d0c2d4ed232751b70034 Sep 09 23:03:42.973462 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:42.973389 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-zmvn9"] Sep 09 23:03:42.979012 ip-10-0-131-92 kubenswrapper[2580]: W0909 23:03:42.978965 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f001595_27ff_43d7_94d2_7ddc18d99671.slice/crio-e151bea20f6d410bdc839422af419f610c7874b25c4df5b63b2428bc406a5cc0 WatchSource:0}: Error finding container e151bea20f6d410bdc839422af419f610c7874b25c4df5b63b2428bc406a5cc0: Status 404 returned error can't find the container with id e151bea20f6d410bdc839422af419f610c7874b25c4df5b63b2428bc406a5cc0 Sep 09 23:03:43.074935 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:43.074889 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b6zvm" event={"ID":"6251fed2-5675-42f5-b0d1-788e1a894092","Type":"ContainerStarted","Data":"322c16066645e0d9359acae20971730f2953133f8163a356bebfa7b05f868c10"} Sep 09 23:03:43.076236 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:43.076208 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-zmvn9" event={"ID":"5f001595-27ff-43d7-94d2-7ddc18d99671","Type":"ContainerStarted","Data":"e151bea20f6d410bdc839422af419f610c7874b25c4df5b63b2428bc406a5cc0"} Sep 09 23:03:43.077512 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:43.077491 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69855d874b-xvwj9" event={"ID":"c15d7152-f7f5-4d4c-8d66-4f955b5b9725","Type":"ContainerStarted","Data":"24614d928ae765956c53ae21a08e1a815c436b81d60c03bf70ffca4a57d42b97"} Sep 09 23:03:43.077615 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:43.077525 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69855d874b-xvwj9" event={"ID":"c15d7152-f7f5-4d4c-8d66-4f955b5b9725","Type":"ContainerStarted","Data":"217972ed9b37df73b542ceb7d6374950a0a40e87d9d1d0c2d4ed232751b70034"} Sep 09 23:03:43.077615 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:43.077613 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-69855d874b-xvwj9" Sep 09 23:03:43.097475 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:43.097420 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-69855d874b-xvwj9" podStartSLOduration=284.097404017 podStartE2EDuration="4m44.097404017s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 22:58:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:03:43.096387558 +0000 UTC m=+284.446254593" watchObservedRunningTime="2026-09-09 23:03:43.097404017 +0000 UTC m=+284.447271049" Sep 09 23:03:45.090514 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:45.090482 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b6zvm" event={"ID":"6251fed2-5675-42f5-b0d1-788e1a894092","Type":"ContainerStarted","Data":"8c1ae64eb08fb9a4bbc6232f47a1f294128fcdce0f6c52187c415f28ad2cc2c5"} Sep 09 23:03:45.090921 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:45.090562 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b6zvm" event={"ID":"6251fed2-5675-42f5-b0d1-788e1a894092","Type":"ContainerStarted","Data":"4f1603296cf29151d23bb3c8d04bf642a6e8f68be20e6ba1dd40de77d47d1a5e"} Sep 09 23:03:45.090921 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:45.090590 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-b6zvm" Sep 09 23:03:45.091950 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:45.091927 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-zmvn9" event={"ID":"5f001595-27ff-43d7-94d2-7ddc18d99671","Type":"ContainerStarted","Data":"8bc1fe79c8b29fd748a48be3fdf18075ba69b0f70164737f37a7cd3751eef26e"} Sep 09 23:03:45.111832 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:45.111781 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-b6zvm" podStartSLOduration=251.194597275 podStartE2EDuration="4m13.111766252s" totalImagesPullingTime="1.917168977s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 22:59:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:03:45.111223583 +0000 UTC m=+286.461090618" watchObservedRunningTime="2026-09-09 23:03:45.111766252 +0000 UTC m=+286.461633284" Sep 09 23:03:45.127252 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:45.127203 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-zmvn9" podStartSLOduration=265.954532361 podStartE2EDuration="4m27.127188662s" totalImagesPullingTime="1.172656301s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 22:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:03:45.126068231 +0000 UTC m=+286.475935265" watchObservedRunningTime="2026-09-09 23:03:45.127188662 +0000 UTC m=+286.477055697" Sep 09 23:03:47.173878 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:47.173844 2580 scope.go:117] "RemoveContainer" containerID="058b36e12be4939a58c04a7bca6e0f7b37f347f9b66c7bd51a7f31aa3069e777" Sep 09 23:03:48.103684 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:48.103655 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xht2x_3b6dd849-6a7e-4141-a460-e9ee0e7ff07c/console-operator/2.log" Sep 09 23:03:48.104065 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:48.104052 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xht2x_3b6dd849-6a7e-4141-a460-e9ee0e7ff07c/console-operator/1.log" Sep 09 23:03:48.104137 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:48.104082 2580 generic.go:356] "Generic (PLEG): container finished" podID="3b6dd849-6a7e-4141-a460-e9ee0e7ff07c" containerID="1e344eebcf8cdd0c0c0e7712513dd51097fae6542b42e7a9152c711a218a5f23" exitCode=255 Sep 09 23:03:48.104137 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:48.104119 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-xht2x" event={"ID":"3b6dd849-6a7e-4141-a460-e9ee0e7ff07c","Type":"ContainerDied","Data":"1e344eebcf8cdd0c0c0e7712513dd51097fae6542b42e7a9152c711a218a5f23"} Sep 09 23:03:48.104203 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:48.104167 2580 scope.go:117] "RemoveContainer" containerID="058b36e12be4939a58c04a7bca6e0f7b37f347f9b66c7bd51a7f31aa3069e777" Sep 09 23:03:48.104482 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:48.104449 2580 scope.go:117] "RemoveContainer" containerID="1e344eebcf8cdd0c0c0e7712513dd51097fae6542b42e7a9152c711a218a5f23" Sep 09 23:03:48.104636 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:48.104617 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-xht2x_openshift-console-operator(3b6dd849-6a7e-4141-a460-e9ee0e7ff07c)\"" pod="openshift-console-operator/console-operator-85b454874b-xht2x" podUID="3b6dd849-6a7e-4141-a460-e9ee0e7ff07c" Sep 09 23:03:48.490666 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:48.490635 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-xht2x" Sep 09 23:03:49.109217 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:49.109193 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xht2x_3b6dd849-6a7e-4141-a460-e9ee0e7ff07c/console-operator/2.log" Sep 09 23:03:49.109520 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:49.109505 2580 scope.go:117] "RemoveContainer" containerID="1e344eebcf8cdd0c0c0e7712513dd51097fae6542b42e7a9152c711a218a5f23" Sep 09 23:03:49.109664 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:49.109647 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-xht2x_openshift-console-operator(3b6dd849-6a7e-4141-a460-e9ee0e7ff07c)\"" pod="openshift-console-operator/console-operator-85b454874b-xht2x" podUID="3b6dd849-6a7e-4141-a460-e9ee0e7ff07c" Sep 09 23:03:50.110022 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:50.109961 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-metrics-certs\") pod \"router-default-74cbccb686-wxl6p\" (UID: \"b7f670bd-13e2-4556-9c0f-ce8317a5ebb5\") " pod="openshift-ingress/router-default-74cbccb686-wxl6p" Sep 09 23:03:50.110488 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:50.110065 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-service-ca-bundle\") pod \"router-default-74cbccb686-wxl6p\" (UID: \"b7f670bd-13e2-4556-9c0f-ce8317a5ebb5\") " pod="openshift-ingress/router-default-74cbccb686-wxl6p" Sep 09 23:03:50.110791 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:50.110769 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-service-ca-bundle\") pod \"router-default-74cbccb686-wxl6p\" (UID: \"b7f670bd-13e2-4556-9c0f-ce8317a5ebb5\") " pod="openshift-ingress/router-default-74cbccb686-wxl6p" Sep 09 23:03:50.112505 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:50.112482 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7f670bd-13e2-4556-9c0f-ce8317a5ebb5-metrics-certs\") pod \"router-default-74cbccb686-wxl6p\" (UID: \"b7f670bd-13e2-4556-9c0f-ce8317a5ebb5\") " pod="openshift-ingress/router-default-74cbccb686-wxl6p" Sep 09 23:03:50.362952 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:50.362861 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-74cbccb686-wxl6p" Sep 09 23:03:50.632949 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:50.632916 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-74cbccb686-wxl6p"] Sep 09 23:03:50.635815 ip-10-0-131-92 kubenswrapper[2580]: W0909 23:03:50.635787 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7f670bd_13e2_4556_9c0f_ce8317a5ebb5.slice/crio-a2a258babd9f0785ee2329ea6abf3489b548cfbdaa2678b83ca01f0c16e940d7 WatchSource:0}: Error finding container a2a258babd9f0785ee2329ea6abf3489b548cfbdaa2678b83ca01f0c16e940d7: Status 404 returned error can't find the container with id a2a258babd9f0785ee2329ea6abf3489b548cfbdaa2678b83ca01f0c16e940d7 Sep 09 23:03:51.116610 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:51.116524 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-74cbccb686-wxl6p" event={"ID":"b7f670bd-13e2-4556-9c0f-ce8317a5ebb5","Type":"ContainerStarted","Data":"3a31577c69e1a77c37807ceb527031b9039d553d8f7edacf1b2aa23157cd01b1"} Sep 09 23:03:51.116610 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:51.116570 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-74cbccb686-wxl6p" event={"ID":"b7f670bd-13e2-4556-9c0f-ce8317a5ebb5","Type":"ContainerStarted","Data":"a2a258babd9f0785ee2329ea6abf3489b548cfbdaa2678b83ca01f0c16e940d7"} Sep 09 23:03:51.136721 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:51.136669 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-74cbccb686-wxl6p" podStartSLOduration=33.136652245 podStartE2EDuration="33.136652245s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:03:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:03:51.135869418 +0000 UTC m=+292.485736447" watchObservedRunningTime="2026-09-09 23:03:51.136652245 +0000 UTC m=+292.486519278" Sep 09 23:03:51.363802 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:51.363762 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-74cbccb686-wxl6p" Sep 09 23:03:51.366146 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:51.366121 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-74cbccb686-wxl6p" Sep 09 23:03:51.998527 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:51.998492 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-xht2x" Sep 09 23:03:51.998867 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:51.998854 2580 scope.go:117] "RemoveContainer" containerID="1e344eebcf8cdd0c0c0e7712513dd51097fae6542b42e7a9152c711a218a5f23" Sep 09 23:03:51.999045 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:51.999026 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-xht2x_openshift-console-operator(3b6dd849-6a7e-4141-a460-e9ee0e7ff07c)\"" pod="openshift-console-operator/console-operator-85b454874b-xht2x" podUID="3b6dd849-6a7e-4141-a460-e9ee0e7ff07c" Sep 09 23:03:52.120212 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:52.120172 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-74cbccb686-wxl6p" Sep 09 23:03:52.121387 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:52.121365 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-74cbccb686-wxl6p" Sep 09 23:03:53.291200 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:53.291158 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-zpssz"] Sep 09 23:03:53.294169 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:53.294151 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-zpssz" Sep 09 23:03:53.296874 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:53.296851 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-46nbv\"" Sep 09 23:03:53.296992 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:53.296858 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Sep 09 23:03:53.305556 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:53.305536 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-zpssz"] Sep 09 23:03:53.437831 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:53.437791 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/275dc765-1345-4219-a0b6-e12119073feb-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-zpssz\" (UID: \"275dc765-1345-4219-a0b6-e12119073feb\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-zpssz" Sep 09 23:03:53.539025 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:53.538949 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/275dc765-1345-4219-a0b6-e12119073feb-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-zpssz\" (UID: \"275dc765-1345-4219-a0b6-e12119073feb\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-zpssz" Sep 09 23:03:53.541323 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:53.541278 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/275dc765-1345-4219-a0b6-e12119073feb-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-zpssz\" (UID: \"275dc765-1345-4219-a0b6-e12119073feb\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-zpssz" Sep 09 23:03:53.602652 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:53.602594 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-zpssz" Sep 09 23:03:53.841227 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:53.841138 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7e149a18-7de7-43d3-89ca-14e842fa6d77-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-86mkf\" (UID: \"7e149a18-7de7-43d3-89ca-14e842fa6d77\") " pod="openshift-insights/insights-runtime-extractor-86mkf" Sep 09 23:03:53.843477 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:53.843445 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7e149a18-7de7-43d3-89ca-14e842fa6d77-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-86mkf\" (UID: \"7e149a18-7de7-43d3-89ca-14e842fa6d77\") " pod="openshift-insights/insights-runtime-extractor-86mkf" Sep 09 23:03:53.859440 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:53.859416 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-86mkf" Sep 09 23:03:53.876161 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:53.876139 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-zpssz"] Sep 09 23:03:53.879139 ip-10-0-131-92 kubenswrapper[2580]: W0909 23:03:53.879118 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod275dc765_1345_4219_a0b6_e12119073feb.slice/crio-22b9a5a353536f588115d3a8af317a8598a47b70a8b1ab5d34dde003512e26fc WatchSource:0}: Error finding container 22b9a5a353536f588115d3a8af317a8598a47b70a8b1ab5d34dde003512e26fc: Status 404 returned error can't find the container with id 22b9a5a353536f588115d3a8af317a8598a47b70a8b1ab5d34dde003512e26fc Sep 09 23:03:54.127029 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:54.126998 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-zpssz" event={"ID":"275dc765-1345-4219-a0b6-e12119073feb","Type":"ContainerStarted","Data":"22b9a5a353536f588115d3a8af317a8598a47b70a8b1ab5d34dde003512e26fc"} Sep 09 23:03:54.130823 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:54.130802 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-86mkf"] Sep 09 23:03:54.133217 ip-10-0-131-92 kubenswrapper[2580]: W0909 23:03:54.133194 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e149a18_7de7_43d3_89ca_14e842fa6d77.slice/crio-bf0c25935b32d796edf6c22c2933398b808508be809e7393c0b5564a9e59a960 WatchSource:0}: Error finding container bf0c25935b32d796edf6c22c2933398b808508be809e7393c0b5564a9e59a960: Status 404 returned error can't find the container with id bf0c25935b32d796edf6c22c2933398b808508be809e7393c0b5564a9e59a960 Sep 09 23:03:54.175323 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:03:54.175294 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-8dwkd" podUID="7cbe663c-0531-4339-b48b-c4d4c3dd5c2b" Sep 09 23:03:55.097921 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:55.097890 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-b6zvm" Sep 09 23:03:55.134517 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:55.134312 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-zpssz" event={"ID":"275dc765-1345-4219-a0b6-e12119073feb","Type":"ContainerStarted","Data":"9af2f408d5b74203f83f3d87696ca4626591b02728dabb072c7f77423e065c44"} Sep 09 23:03:55.134517 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:55.134377 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-zpssz" Sep 09 23:03:55.136653 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:55.136610 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-86mkf" event={"ID":"7e149a18-7de7-43d3-89ca-14e842fa6d77","Type":"ContainerStarted","Data":"af2838a1362c7276897fd700285df910c88996e508e642c2ee7ccd86c2bcb171"} Sep 09 23:03:55.136798 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:55.136668 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-86mkf" event={"ID":"7e149a18-7de7-43d3-89ca-14e842fa6d77","Type":"ContainerStarted","Data":"bf0c25935b32d796edf6c22c2933398b808508be809e7393c0b5564a9e59a960"} Sep 09 23:03:55.140102 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:55.140080 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-zpssz" Sep 09 23:03:55.149710 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:55.149643 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-zpssz" podStartSLOduration=1.042308326 podStartE2EDuration="2.149619095s" totalImagesPullingTime="1.107310769s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:03:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:03:55.149487166 +0000 UTC m=+296.499354199" watchObservedRunningTime="2026-09-09 23:03:55.149619095 +0000 UTC m=+296.499486129" Sep 09 23:03:55.355504 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:55.355462 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-m7l6j"] Sep 09 23:03:55.358658 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:55.358636 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-m7l6j" Sep 09 23:03:55.361446 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:55.361423 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-29lt5\"" Sep 09 23:03:55.361736 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:55.361716 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 09 23:03:55.361836 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:55.361769 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 09 23:03:55.362072 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:55.362057 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Sep 09 23:03:55.362145 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:55.362071 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 09 23:03:55.362145 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:55.362105 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Sep 09 23:03:55.371784 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:55.371760 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-m7l6j"] Sep 09 23:03:55.455234 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:55.455204 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32c1d310-58b7-48ce-a72e-bd2dd4b64ab4-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-m7l6j\" (UID: \"32c1d310-58b7-48ce-a72e-bd2dd4b64ab4\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-m7l6j" Sep 09 23:03:55.455403 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:55.455244 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/32c1d310-58b7-48ce-a72e-bd2dd4b64ab4-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-m7l6j\" (UID: \"32c1d310-58b7-48ce-a72e-bd2dd4b64ab4\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-m7l6j" Sep 09 23:03:55.455403 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:55.455290 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntzs5\" (UniqueName: \"kubernetes.io/projected/32c1d310-58b7-48ce-a72e-bd2dd4b64ab4-kube-api-access-ntzs5\") pod \"prometheus-operator-6f4946db56-m7l6j\" (UID: \"32c1d310-58b7-48ce-a72e-bd2dd4b64ab4\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-m7l6j" Sep 09 23:03:55.455403 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:55.455355 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/32c1d310-58b7-48ce-a72e-bd2dd4b64ab4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-m7l6j\" (UID: \"32c1d310-58b7-48ce-a72e-bd2dd4b64ab4\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-m7l6j" Sep 09 23:03:55.556268 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:55.556186 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntzs5\" (UniqueName: \"kubernetes.io/projected/32c1d310-58b7-48ce-a72e-bd2dd4b64ab4-kube-api-access-ntzs5\") pod \"prometheus-operator-6f4946db56-m7l6j\" (UID: \"32c1d310-58b7-48ce-a72e-bd2dd4b64ab4\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-m7l6j" Sep 09 23:03:55.556268 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:55.556224 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/32c1d310-58b7-48ce-a72e-bd2dd4b64ab4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-m7l6j\" (UID: \"32c1d310-58b7-48ce-a72e-bd2dd4b64ab4\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-m7l6j" Sep 09 23:03:55.556450 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:55.556348 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32c1d310-58b7-48ce-a72e-bd2dd4b64ab4-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-m7l6j\" (UID: \"32c1d310-58b7-48ce-a72e-bd2dd4b64ab4\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-m7l6j" Sep 09 23:03:55.556450 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:55.556397 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/32c1d310-58b7-48ce-a72e-bd2dd4b64ab4-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-m7l6j\" (UID: \"32c1d310-58b7-48ce-a72e-bd2dd4b64ab4\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-m7l6j" Sep 09 23:03:55.556945 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:55.556922 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32c1d310-58b7-48ce-a72e-bd2dd4b64ab4-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-m7l6j\" (UID: \"32c1d310-58b7-48ce-a72e-bd2dd4b64ab4\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-m7l6j" Sep 09 23:03:55.558643 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:55.558619 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/32c1d310-58b7-48ce-a72e-bd2dd4b64ab4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-m7l6j\" (UID: \"32c1d310-58b7-48ce-a72e-bd2dd4b64ab4\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-m7l6j" Sep 09 23:03:55.558723 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:55.558685 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/32c1d310-58b7-48ce-a72e-bd2dd4b64ab4-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-m7l6j\" (UID: \"32c1d310-58b7-48ce-a72e-bd2dd4b64ab4\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-m7l6j" Sep 09 23:03:55.564748 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:55.564726 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntzs5\" (UniqueName: \"kubernetes.io/projected/32c1d310-58b7-48ce-a72e-bd2dd4b64ab4-kube-api-access-ntzs5\") pod \"prometheus-operator-6f4946db56-m7l6j\" (UID: \"32c1d310-58b7-48ce-a72e-bd2dd4b64ab4\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-m7l6j" Sep 09 23:03:55.666943 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:55.666907 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-m7l6j" Sep 09 23:03:55.938594 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:55.938558 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-m7l6j"] Sep 09 23:03:55.941916 ip-10-0-131-92 kubenswrapper[2580]: W0909 23:03:55.941889 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32c1d310_58b7_48ce_a72e_bd2dd4b64ab4.slice/crio-da07868dad2c94535cc734e47dcf861719073d61bbf4a5b50f7c3fb5078e15c6 WatchSource:0}: Error finding container da07868dad2c94535cc734e47dcf861719073d61bbf4a5b50f7c3fb5078e15c6: Status 404 returned error can't find the container with id da07868dad2c94535cc734e47dcf861719073d61bbf4a5b50f7c3fb5078e15c6 Sep 09 23:03:56.142809 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:56.142773 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-86mkf" event={"ID":"7e149a18-7de7-43d3-89ca-14e842fa6d77","Type":"ContainerStarted","Data":"5290197fc4b2ef4cf1bd267256e45a199eebf0af5a8ac9b37525187f500ac863"} Sep 09 23:03:56.144024 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:56.143996 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-m7l6j" event={"ID":"32c1d310-58b7-48ce-a72e-bd2dd4b64ab4","Type":"ContainerStarted","Data":"da07868dad2c94535cc734e47dcf861719073d61bbf4a5b50f7c3fb5078e15c6"} Sep 09 23:03:57.149252 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:57.149210 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-86mkf" event={"ID":"7e149a18-7de7-43d3-89ca-14e842fa6d77","Type":"ContainerStarted","Data":"5c00303ba60d808379bf3b8c33a887a6707f9d4cb91cb9df2af64b27fd4c45fd"} Sep 09 23:03:57.168749 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:57.168696 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-86mkf" podStartSLOduration=33.839391075 podStartE2EDuration="36.168677751s" totalImagesPullingTime="2.329286676s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 23:03:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:03:57.166862493 +0000 UTC m=+298.516729541" watchObservedRunningTime="2026-09-09 23:03:57.168677751 +0000 UTC m=+298.518544784" Sep 09 23:03:58.155161 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:58.155120 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-m7l6j" event={"ID":"32c1d310-58b7-48ce-a72e-bd2dd4b64ab4","Type":"ContainerStarted","Data":"a38cd9623228193253d4e064203637218bdd44a24aceccc1fe3ae8eb5b0b96ca"} Sep 09 23:03:58.155506 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:58.155181 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-m7l6j" event={"ID":"32c1d310-58b7-48ce-a72e-bd2dd4b64ab4","Type":"ContainerStarted","Data":"56741960ea53fe1979140b0cc94ff00041cf374ced043a3e7eea34e7a58dceb9"} Sep 09 23:03:58.172161 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:58.172115 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-m7l6j" podStartSLOduration=1.63475104 podStartE2EDuration="3.172102094s" totalImagesPullingTime="1.537351054s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:03:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:03:58.170821844 +0000 UTC m=+299.520688878" watchObservedRunningTime="2026-09-09 23:03:58.172102094 +0000 UTC m=+299.521969128" Sep 09 23:03:59.079512 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.079484 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xht2x_3b6dd849-6a7e-4141-a460-e9ee0e7ff07c/console-operator/2.log" Sep 09 23:03:59.080335 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.080302 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xht2x_3b6dd849-6a7e-4141-a460-e9ee0e7ff07c/console-operator/2.log" Sep 09 23:03:59.090153 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.090131 2580 kubelet.go:1648] "Image garbage collection succeeded" Sep 09 23:03:59.687191 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.687155 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-nw5xf"] Sep 09 23:03:59.690480 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.690462 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-nw5xf" Sep 09 23:03:59.695005 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.694967 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-svb96\"" Sep 09 23:03:59.696030 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.696012 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Sep 09 23:03:59.696141 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.696012 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Sep 09 23:03:59.700212 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.700191 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-nw5xf"] Sep 09 23:03:59.722177 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.722143 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-gbt9m"] Sep 09 23:03:59.725340 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.725325 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-gbt9m" Sep 09 23:03:59.727692 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.727667 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-wqs6f"] Sep 09 23:03:59.728283 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.728263 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-259kg\"" Sep 09 23:03:59.728397 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.728267 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Sep 09 23:03:59.728397 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.728286 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Sep 09 23:03:59.728494 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.728410 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Sep 09 23:03:59.730850 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.730831 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wqs6f" Sep 09 23:03:59.734946 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.734929 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-jc9mn\"" Sep 09 23:03:59.735078 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.734933 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 09 23:03:59.735219 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.735199 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 09 23:03:59.735270 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.735226 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 09 23:03:59.742804 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.742783 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-gbt9m"] Sep 09 23:03:59.789888 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.789856 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/92e597ee-8c57-4c77-908e-6e8045ffcded-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-nw5xf\" (UID: \"92e597ee-8c57-4c77-908e-6e8045ffcded\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-nw5xf" Sep 09 23:03:59.790109 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.789895 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/92e597ee-8c57-4c77-908e-6e8045ffcded-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-nw5xf\" (UID: \"92e597ee-8c57-4c77-908e-6e8045ffcded\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-nw5xf" Sep 09 23:03:59.790109 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.789930 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/92e597ee-8c57-4c77-908e-6e8045ffcded-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-nw5xf\" (UID: \"92e597ee-8c57-4c77-908e-6e8045ffcded\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-nw5xf" Sep 09 23:03:59.790109 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.790017 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpnrb\" (UniqueName: \"kubernetes.io/projected/92e597ee-8c57-4c77-908e-6e8045ffcded-kube-api-access-mpnrb\") pod \"openshift-state-metrics-657fd6c54-nw5xf\" (UID: \"92e597ee-8c57-4c77-908e-6e8045ffcded\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-nw5xf" Sep 09 23:03:59.891015 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.890953 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpnrb\" (UniqueName: \"kubernetes.io/projected/92e597ee-8c57-4c77-908e-6e8045ffcded-kube-api-access-mpnrb\") pod \"openshift-state-metrics-657fd6c54-nw5xf\" (UID: \"92e597ee-8c57-4c77-908e-6e8045ffcded\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-nw5xf" Sep 09 23:03:59.891207 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.891045 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrq9n\" (UniqueName: \"kubernetes.io/projected/caacaea7-7d91-480b-b93a-e50c9a7f1d25-kube-api-access-jrq9n\") pod \"kube-state-metrics-74cb57df7-gbt9m\" (UID: \"caacaea7-7d91-480b-b93a-e50c9a7f1d25\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-gbt9m" Sep 09 23:03:59.891207 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.891080 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e2231319-641c-40bb-901b-2577df536ab0-sys\") pod \"node-exporter-wqs6f\" (UID: \"e2231319-641c-40bb-901b-2577df536ab0\") " pod="openshift-monitoring/node-exporter-wqs6f" Sep 09 23:03:59.891207 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.891110 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2231319-641c-40bb-901b-2577df536ab0-metrics-client-ca\") pod \"node-exporter-wqs6f\" (UID: \"e2231319-641c-40bb-901b-2577df536ab0\") " pod="openshift-monitoring/node-exporter-wqs6f" Sep 09 23:03:59.891207 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.891164 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/caacaea7-7d91-480b-b93a-e50c9a7f1d25-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-gbt9m\" (UID: \"caacaea7-7d91-480b-b93a-e50c9a7f1d25\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-gbt9m" Sep 09 23:03:59.891207 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.891195 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e2231319-641c-40bb-901b-2577df536ab0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wqs6f\" (UID: \"e2231319-641c-40bb-901b-2577df536ab0\") " pod="openshift-monitoring/node-exporter-wqs6f" Sep 09 23:03:59.891395 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.891228 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/92e597ee-8c57-4c77-908e-6e8045ffcded-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-nw5xf\" (UID: \"92e597ee-8c57-4c77-908e-6e8045ffcded\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-nw5xf" Sep 09 23:03:59.891395 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.891306 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/caacaea7-7d91-480b-b93a-e50c9a7f1d25-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-gbt9m\" (UID: \"caacaea7-7d91-480b-b93a-e50c9a7f1d25\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-gbt9m" Sep 09 23:03:59.891395 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.891338 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7pk4\" (UniqueName: \"kubernetes.io/projected/e2231319-641c-40bb-901b-2577df536ab0-kube-api-access-t7pk4\") pod \"node-exporter-wqs6f\" (UID: \"e2231319-641c-40bb-901b-2577df536ab0\") " pod="openshift-monitoring/node-exporter-wqs6f" Sep 09 23:03:59.891395 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.891363 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/caacaea7-7d91-480b-b93a-e50c9a7f1d25-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-gbt9m\" (UID: \"caacaea7-7d91-480b-b93a-e50c9a7f1d25\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-gbt9m" Sep 09 23:03:59.891395 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.891392 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/92e597ee-8c57-4c77-908e-6e8045ffcded-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-nw5xf\" (UID: \"92e597ee-8c57-4c77-908e-6e8045ffcded\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-nw5xf" Sep 09 23:03:59.891633 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.891427 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e2231319-641c-40bb-901b-2577df536ab0-node-exporter-textfile\") pod \"node-exporter-wqs6f\" (UID: \"e2231319-641c-40bb-901b-2577df536ab0\") " pod="openshift-monitoring/node-exporter-wqs6f" Sep 09 23:03:59.891633 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.891444 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e2231319-641c-40bb-901b-2577df536ab0-node-exporter-wtmp\") pod \"node-exporter-wqs6f\" (UID: \"e2231319-641c-40bb-901b-2577df536ab0\") " pod="openshift-monitoring/node-exporter-wqs6f" Sep 09 23:03:59.891633 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.891464 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e2231319-641c-40bb-901b-2577df536ab0-root\") pod \"node-exporter-wqs6f\" (UID: \"e2231319-641c-40bb-901b-2577df536ab0\") " pod="openshift-monitoring/node-exporter-wqs6f" Sep 09 23:03:59.891633 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.891500 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e2231319-641c-40bb-901b-2577df536ab0-node-exporter-accelerators-collector-config\") pod \"node-exporter-wqs6f\" (UID: \"e2231319-641c-40bb-901b-2577df536ab0\") " pod="openshift-monitoring/node-exporter-wqs6f" Sep 09 23:03:59.891633 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.891530 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e2231319-641c-40bb-901b-2577df536ab0-node-exporter-tls\") pod \"node-exporter-wqs6f\" (UID: \"e2231319-641c-40bb-901b-2577df536ab0\") " pod="openshift-monitoring/node-exporter-wqs6f" Sep 09 23:03:59.891633 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.891572 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/caacaea7-7d91-480b-b93a-e50c9a7f1d25-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-gbt9m\" (UID: \"caacaea7-7d91-480b-b93a-e50c9a7f1d25\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-gbt9m" Sep 09 23:03:59.891633 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.891621 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/92e597ee-8c57-4c77-908e-6e8045ffcded-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-nw5xf\" (UID: \"92e597ee-8c57-4c77-908e-6e8045ffcded\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-nw5xf" Sep 09 23:03:59.891912 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.891651 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/caacaea7-7d91-480b-b93a-e50c9a7f1d25-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-gbt9m\" (UID: \"caacaea7-7d91-480b-b93a-e50c9a7f1d25\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-gbt9m" Sep 09 23:03:59.891912 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.891787 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/92e597ee-8c57-4c77-908e-6e8045ffcded-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-nw5xf\" (UID: \"92e597ee-8c57-4c77-908e-6e8045ffcded\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-nw5xf" Sep 09 23:03:59.893762 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.893741 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/92e597ee-8c57-4c77-908e-6e8045ffcded-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-nw5xf\" (UID: \"92e597ee-8c57-4c77-908e-6e8045ffcded\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-nw5xf" Sep 09 23:03:59.893762 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.893755 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/92e597ee-8c57-4c77-908e-6e8045ffcded-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-nw5xf\" (UID: \"92e597ee-8c57-4c77-908e-6e8045ffcded\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-nw5xf" Sep 09 23:03:59.900061 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.900039 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpnrb\" (UniqueName: \"kubernetes.io/projected/92e597ee-8c57-4c77-908e-6e8045ffcded-kube-api-access-mpnrb\") pod \"openshift-state-metrics-657fd6c54-nw5xf\" (UID: \"92e597ee-8c57-4c77-908e-6e8045ffcded\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-nw5xf" Sep 09 23:03:59.992409 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.992326 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e2231319-641c-40bb-901b-2577df536ab0-node-exporter-textfile\") pod \"node-exporter-wqs6f\" (UID: \"e2231319-641c-40bb-901b-2577df536ab0\") " pod="openshift-monitoring/node-exporter-wqs6f" Sep 09 23:03:59.992409 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.992364 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e2231319-641c-40bb-901b-2577df536ab0-node-exporter-wtmp\") pod \"node-exporter-wqs6f\" (UID: \"e2231319-641c-40bb-901b-2577df536ab0\") " pod="openshift-monitoring/node-exporter-wqs6f" Sep 09 23:03:59.992409 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.992382 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e2231319-641c-40bb-901b-2577df536ab0-root\") pod \"node-exporter-wqs6f\" (UID: \"e2231319-641c-40bb-901b-2577df536ab0\") " pod="openshift-monitoring/node-exporter-wqs6f" Sep 09 23:03:59.992409 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.992400 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e2231319-641c-40bb-901b-2577df536ab0-node-exporter-accelerators-collector-config\") pod \"node-exporter-wqs6f\" (UID: \"e2231319-641c-40bb-901b-2577df536ab0\") " pod="openshift-monitoring/node-exporter-wqs6f" Sep 09 23:03:59.992709 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.992422 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e2231319-641c-40bb-901b-2577df536ab0-node-exporter-tls\") pod \"node-exporter-wqs6f\" (UID: \"e2231319-641c-40bb-901b-2577df536ab0\") " pod="openshift-monitoring/node-exporter-wqs6f" Sep 09 23:03:59.992709 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.992449 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/caacaea7-7d91-480b-b93a-e50c9a7f1d25-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-gbt9m\" (UID: \"caacaea7-7d91-480b-b93a-e50c9a7f1d25\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-gbt9m" Sep 09 23:03:59.992709 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.992477 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/caacaea7-7d91-480b-b93a-e50c9a7f1d25-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-gbt9m\" (UID: \"caacaea7-7d91-480b-b93a-e50c9a7f1d25\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-gbt9m" Sep 09 23:03:59.992709 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.992479 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e2231319-641c-40bb-901b-2577df536ab0-root\") pod \"node-exporter-wqs6f\" (UID: \"e2231319-641c-40bb-901b-2577df536ab0\") " pod="openshift-monitoring/node-exporter-wqs6f" Sep 09 23:03:59.992709 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.992518 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrq9n\" (UniqueName: \"kubernetes.io/projected/caacaea7-7d91-480b-b93a-e50c9a7f1d25-kube-api-access-jrq9n\") pod \"kube-state-metrics-74cb57df7-gbt9m\" (UID: \"caacaea7-7d91-480b-b93a-e50c9a7f1d25\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-gbt9m" Sep 09 23:03:59.992709 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.992527 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e2231319-641c-40bb-901b-2577df536ab0-node-exporter-wtmp\") pod \"node-exporter-wqs6f\" (UID: \"e2231319-641c-40bb-901b-2577df536ab0\") " pod="openshift-monitoring/node-exporter-wqs6f" Sep 09 23:03:59.992709 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.992549 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e2231319-641c-40bb-901b-2577df536ab0-sys\") pod \"node-exporter-wqs6f\" (UID: \"e2231319-641c-40bb-901b-2577df536ab0\") " pod="openshift-monitoring/node-exporter-wqs6f" Sep 09 23:03:59.992709 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.992576 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2231319-641c-40bb-901b-2577df536ab0-metrics-client-ca\") pod \"node-exporter-wqs6f\" (UID: \"e2231319-641c-40bb-901b-2577df536ab0\") " pod="openshift-monitoring/node-exporter-wqs6f" Sep 09 23:03:59.992709 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.992603 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/caacaea7-7d91-480b-b93a-e50c9a7f1d25-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-gbt9m\" (UID: \"caacaea7-7d91-480b-b93a-e50c9a7f1d25\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-gbt9m" Sep 09 23:03:59.992709 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.992626 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e2231319-641c-40bb-901b-2577df536ab0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wqs6f\" (UID: \"e2231319-641c-40bb-901b-2577df536ab0\") " pod="openshift-monitoring/node-exporter-wqs6f" Sep 09 23:03:59.992709 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.992646 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/caacaea7-7d91-480b-b93a-e50c9a7f1d25-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-gbt9m\" (UID: \"caacaea7-7d91-480b-b93a-e50c9a7f1d25\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-gbt9m" Sep 09 23:03:59.992709 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.992667 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e2231319-641c-40bb-901b-2577df536ab0-sys\") pod \"node-exporter-wqs6f\" (UID: \"e2231319-641c-40bb-901b-2577df536ab0\") " pod="openshift-monitoring/node-exporter-wqs6f" Sep 09 23:03:59.992709 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.992673 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7pk4\" (UniqueName: \"kubernetes.io/projected/e2231319-641c-40bb-901b-2577df536ab0-kube-api-access-t7pk4\") pod \"node-exporter-wqs6f\" (UID: \"e2231319-641c-40bb-901b-2577df536ab0\") " pod="openshift-monitoring/node-exporter-wqs6f" Sep 09 23:03:59.993357 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.992742 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/caacaea7-7d91-480b-b93a-e50c9a7f1d25-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-gbt9m\" (UID: \"caacaea7-7d91-480b-b93a-e50c9a7f1d25\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-gbt9m" Sep 09 23:03:59.993357 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.992762 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e2231319-641c-40bb-901b-2577df536ab0-node-exporter-textfile\") pod \"node-exporter-wqs6f\" (UID: \"e2231319-641c-40bb-901b-2577df536ab0\") " pod="openshift-monitoring/node-exporter-wqs6f" Sep 09 23:03:59.993357 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.993160 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/caacaea7-7d91-480b-b93a-e50c9a7f1d25-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-gbt9m\" (UID: \"caacaea7-7d91-480b-b93a-e50c9a7f1d25\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-gbt9m" Sep 09 23:03:59.993357 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.993162 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e2231319-641c-40bb-901b-2577df536ab0-node-exporter-accelerators-collector-config\") pod \"node-exporter-wqs6f\" (UID: \"e2231319-641c-40bb-901b-2577df536ab0\") " pod="openshift-monitoring/node-exporter-wqs6f" Sep 09 23:03:59.993357 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.993271 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/caacaea7-7d91-480b-b93a-e50c9a7f1d25-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-gbt9m\" (UID: \"caacaea7-7d91-480b-b93a-e50c9a7f1d25\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-gbt9m" Sep 09 23:03:59.993610 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.993361 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2231319-641c-40bb-901b-2577df536ab0-metrics-client-ca\") pod \"node-exporter-wqs6f\" (UID: \"e2231319-641c-40bb-901b-2577df536ab0\") " pod="openshift-monitoring/node-exporter-wqs6f" Sep 09 23:03:59.993610 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.993516 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/caacaea7-7d91-480b-b93a-e50c9a7f1d25-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-gbt9m\" (UID: \"caacaea7-7d91-480b-b93a-e50c9a7f1d25\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-gbt9m" Sep 09 23:03:59.995060 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.995038 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e2231319-641c-40bb-901b-2577df536ab0-node-exporter-tls\") pod \"node-exporter-wqs6f\" (UID: \"e2231319-641c-40bb-901b-2577df536ab0\") " pod="openshift-monitoring/node-exporter-wqs6f" Sep 09 23:03:59.995269 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.995220 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/caacaea7-7d91-480b-b93a-e50c9a7f1d25-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-gbt9m\" (UID: \"caacaea7-7d91-480b-b93a-e50c9a7f1d25\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-gbt9m" Sep 09 23:03:59.995318 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.995274 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e2231319-641c-40bb-901b-2577df536ab0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wqs6f\" (UID: \"e2231319-641c-40bb-901b-2577df536ab0\") " pod="openshift-monitoring/node-exporter-wqs6f" Sep 09 23:03:59.995566 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.995548 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/caacaea7-7d91-480b-b93a-e50c9a7f1d25-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-gbt9m\" (UID: \"caacaea7-7d91-480b-b93a-e50c9a7f1d25\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-gbt9m" Sep 09 23:03:59.999007 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:03:59.998989 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-nw5xf" Sep 09 23:04:00.001230 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:00.001207 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrq9n\" (UniqueName: \"kubernetes.io/projected/caacaea7-7d91-480b-b93a-e50c9a7f1d25-kube-api-access-jrq9n\") pod \"kube-state-metrics-74cb57df7-gbt9m\" (UID: \"caacaea7-7d91-480b-b93a-e50c9a7f1d25\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-gbt9m" Sep 09 23:04:00.001766 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:00.001747 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7pk4\" (UniqueName: \"kubernetes.io/projected/e2231319-641c-40bb-901b-2577df536ab0-kube-api-access-t7pk4\") pod \"node-exporter-wqs6f\" (UID: \"e2231319-641c-40bb-901b-2577df536ab0\") " pod="openshift-monitoring/node-exporter-wqs6f" Sep 09 23:04:00.035082 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:00.034382 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-gbt9m" Sep 09 23:04:00.041174 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:00.041149 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wqs6f" Sep 09 23:04:00.053548 ip-10-0-131-92 kubenswrapper[2580]: W0909 23:04:00.053513 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2231319_641c_40bb_901b_2577df536ab0.slice/crio-10489178a30fd0060b19a5794b6e47be51074bb4d410f489696e926cb5be44b9 WatchSource:0}: Error finding container 10489178a30fd0060b19a5794b6e47be51074bb4d410f489696e926cb5be44b9: Status 404 returned error can't find the container with id 10489178a30fd0060b19a5794b6e47be51074bb4d410f489696e926cb5be44b9 Sep 09 23:04:00.163120 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:00.163081 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wqs6f" event={"ID":"e2231319-641c-40bb-901b-2577df536ab0","Type":"ContainerStarted","Data":"10489178a30fd0060b19a5794b6e47be51074bb4d410f489696e926cb5be44b9"} Sep 09 23:04:00.266314 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:00.266238 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-nw5xf"] Sep 09 23:04:00.268960 ip-10-0-131-92 kubenswrapper[2580]: W0909 23:04:00.268937 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92e597ee_8c57_4c77_908e_6e8045ffcded.slice/crio-0fbeaf9e26ccac42c00a241a558394710f30d72ec6852106f12e9e27aaee6f1f WatchSource:0}: Error finding container 0fbeaf9e26ccac42c00a241a558394710f30d72ec6852106f12e9e27aaee6f1f: Status 404 returned error can't find the container with id 0fbeaf9e26ccac42c00a241a558394710f30d72ec6852106f12e9e27aaee6f1f Sep 09 23:04:00.313354 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:00.313329 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-gbt9m"] Sep 09 23:04:00.316615 ip-10-0-131-92 kubenswrapper[2580]: W0909 23:04:00.316591 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaacaea7_7d91_480b_b93a_e50c9a7f1d25.slice/crio-f32a2316b82918ded0dff150dcb9f474d53f394c9752ed6406973ed24f9ef246 WatchSource:0}: Error finding container f32a2316b82918ded0dff150dcb9f474d53f394c9752ed6406973ed24f9ef246: Status 404 returned error can't find the container with id f32a2316b82918ded0dff150dcb9f474d53f394c9752ed6406973ed24f9ef246 Sep 09 23:04:00.388318 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:00.388295 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 23:04:01.170045 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:01.170005 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-nw5xf" event={"ID":"92e597ee-8c57-4c77-908e-6e8045ffcded","Type":"ContainerStarted","Data":"10994b511ffc8167cba42d8765a670009b148950e52bb1ecff3a233cb0876941"} Sep 09 23:04:01.170494 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:01.170089 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-nw5xf" event={"ID":"92e597ee-8c57-4c77-908e-6e8045ffcded","Type":"ContainerStarted","Data":"c595401158d0d0ce46608221412d6d9369ed12c3e71b637163fa675e03dd59d9"} Sep 09 23:04:01.170494 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:01.170120 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-nw5xf" event={"ID":"92e597ee-8c57-4c77-908e-6e8045ffcded","Type":"ContainerStarted","Data":"0fbeaf9e26ccac42c00a241a558394710f30d72ec6852106f12e9e27aaee6f1f"} Sep 09 23:04:01.171847 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:01.171816 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-gbt9m" event={"ID":"caacaea7-7d91-480b-b93a-e50c9a7f1d25","Type":"ContainerStarted","Data":"f32a2316b82918ded0dff150dcb9f474d53f394c9752ed6406973ed24f9ef246"} Sep 09 23:04:01.702803 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:01.702761 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7f85c99747-bcwgk"] Sep 09 23:04:01.707420 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:01.707387 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7f85c99747-bcwgk" Sep 09 23:04:01.710501 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:01.710470 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Sep 09 23:04:01.710748 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:01.710732 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-33it8blj2q2d8\"" Sep 09 23:04:01.711822 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:01.711285 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Sep 09 23:04:01.711822 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:01.711410 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Sep 09 23:04:01.711822 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:01.711511 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-tm5dg\"" Sep 09 23:04:01.711822 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:01.711677 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Sep 09 23:04:01.711822 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:01.711689 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Sep 09 23:04:01.720681 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:01.720652 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7f85c99747-bcwgk"] Sep 09 23:04:01.805828 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:01.805793 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7106047f-cae5-4ae2-8de3-1b71715187a1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7f85c99747-bcwgk\" (UID: \"7106047f-cae5-4ae2-8de3-1b71715187a1\") " pod="openshift-monitoring/thanos-querier-7f85c99747-bcwgk" Sep 09 23:04:01.805828 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:01.805844 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7106047f-cae5-4ae2-8de3-1b71715187a1-metrics-client-ca\") pod \"thanos-querier-7f85c99747-bcwgk\" (UID: \"7106047f-cae5-4ae2-8de3-1b71715187a1\") " pod="openshift-monitoring/thanos-querier-7f85c99747-bcwgk" Sep 09 23:04:01.806066 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:01.805942 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/7106047f-cae5-4ae2-8de3-1b71715187a1-secret-thanos-querier-tls\") pod \"thanos-querier-7f85c99747-bcwgk\" (UID: \"7106047f-cae5-4ae2-8de3-1b71715187a1\") " pod="openshift-monitoring/thanos-querier-7f85c99747-bcwgk" Sep 09 23:04:01.806066 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:01.805995 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txp9p\" (UniqueName: \"kubernetes.io/projected/7106047f-cae5-4ae2-8de3-1b71715187a1-kube-api-access-txp9p\") pod \"thanos-querier-7f85c99747-bcwgk\" (UID: \"7106047f-cae5-4ae2-8de3-1b71715187a1\") " pod="openshift-monitoring/thanos-querier-7f85c99747-bcwgk" Sep 09 23:04:01.806066 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:01.806019 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7106047f-cae5-4ae2-8de3-1b71715187a1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7f85c99747-bcwgk\" (UID: \"7106047f-cae5-4ae2-8de3-1b71715187a1\") " pod="openshift-monitoring/thanos-querier-7f85c99747-bcwgk" Sep 09 23:04:01.806066 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:01.806039 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7106047f-cae5-4ae2-8de3-1b71715187a1-secret-grpc-tls\") pod \"thanos-querier-7f85c99747-bcwgk\" (UID: \"7106047f-cae5-4ae2-8de3-1b71715187a1\") " pod="openshift-monitoring/thanos-querier-7f85c99747-bcwgk" Sep 09 23:04:01.806213 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:01.806114 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/7106047f-cae5-4ae2-8de3-1b71715187a1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7f85c99747-bcwgk\" (UID: \"7106047f-cae5-4ae2-8de3-1b71715187a1\") " pod="openshift-monitoring/thanos-querier-7f85c99747-bcwgk" Sep 09 23:04:01.806213 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:01.806175 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/7106047f-cae5-4ae2-8de3-1b71715187a1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7f85c99747-bcwgk\" (UID: \"7106047f-cae5-4ae2-8de3-1b71715187a1\") " pod="openshift-monitoring/thanos-querier-7f85c99747-bcwgk" Sep 09 23:04:01.906883 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:01.906841 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txp9p\" (UniqueName: \"kubernetes.io/projected/7106047f-cae5-4ae2-8de3-1b71715187a1-kube-api-access-txp9p\") pod \"thanos-querier-7f85c99747-bcwgk\" (UID: \"7106047f-cae5-4ae2-8de3-1b71715187a1\") " pod="openshift-monitoring/thanos-querier-7f85c99747-bcwgk" Sep 09 23:04:01.906883 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:01.906885 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7106047f-cae5-4ae2-8de3-1b71715187a1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7f85c99747-bcwgk\" (UID: \"7106047f-cae5-4ae2-8de3-1b71715187a1\") " pod="openshift-monitoring/thanos-querier-7f85c99747-bcwgk" Sep 09 23:04:01.907151 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:01.906903 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7106047f-cae5-4ae2-8de3-1b71715187a1-secret-grpc-tls\") pod \"thanos-querier-7f85c99747-bcwgk\" (UID: \"7106047f-cae5-4ae2-8de3-1b71715187a1\") " pod="openshift-monitoring/thanos-querier-7f85c99747-bcwgk" Sep 09 23:04:01.907151 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:01.906948 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/7106047f-cae5-4ae2-8de3-1b71715187a1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7f85c99747-bcwgk\" (UID: \"7106047f-cae5-4ae2-8de3-1b71715187a1\") " pod="openshift-monitoring/thanos-querier-7f85c99747-bcwgk" Sep 09 23:04:01.907151 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:01.906998 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/7106047f-cae5-4ae2-8de3-1b71715187a1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7f85c99747-bcwgk\" (UID: \"7106047f-cae5-4ae2-8de3-1b71715187a1\") " pod="openshift-monitoring/thanos-querier-7f85c99747-bcwgk" Sep 09 23:04:01.907151 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:01.907029 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7106047f-cae5-4ae2-8de3-1b71715187a1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7f85c99747-bcwgk\" (UID: \"7106047f-cae5-4ae2-8de3-1b71715187a1\") " pod="openshift-monitoring/thanos-querier-7f85c99747-bcwgk" Sep 09 23:04:01.907151 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:01.907067 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7106047f-cae5-4ae2-8de3-1b71715187a1-metrics-client-ca\") pod \"thanos-querier-7f85c99747-bcwgk\" (UID: \"7106047f-cae5-4ae2-8de3-1b71715187a1\") " pod="openshift-monitoring/thanos-querier-7f85c99747-bcwgk" Sep 09 23:04:01.907151 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:01.907131 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/7106047f-cae5-4ae2-8de3-1b71715187a1-secret-thanos-querier-tls\") pod \"thanos-querier-7f85c99747-bcwgk\" (UID: \"7106047f-cae5-4ae2-8de3-1b71715187a1\") " pod="openshift-monitoring/thanos-querier-7f85c99747-bcwgk" Sep 09 23:04:01.907963 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:01.907930 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7106047f-cae5-4ae2-8de3-1b71715187a1-metrics-client-ca\") pod \"thanos-querier-7f85c99747-bcwgk\" (UID: \"7106047f-cae5-4ae2-8de3-1b71715187a1\") " pod="openshift-monitoring/thanos-querier-7f85c99747-bcwgk" Sep 09 23:04:01.909558 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:01.909528 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7106047f-cae5-4ae2-8de3-1b71715187a1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7f85c99747-bcwgk\" (UID: \"7106047f-cae5-4ae2-8de3-1b71715187a1\") " pod="openshift-monitoring/thanos-querier-7f85c99747-bcwgk" Sep 09 23:04:01.909659 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:01.909586 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/7106047f-cae5-4ae2-8de3-1b71715187a1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7f85c99747-bcwgk\" (UID: \"7106047f-cae5-4ae2-8de3-1b71715187a1\") " pod="openshift-monitoring/thanos-querier-7f85c99747-bcwgk" Sep 09 23:04:01.909819 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:01.909800 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/7106047f-cae5-4ae2-8de3-1b71715187a1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7f85c99747-bcwgk\" (UID: \"7106047f-cae5-4ae2-8de3-1b71715187a1\") " pod="openshift-monitoring/thanos-querier-7f85c99747-bcwgk" Sep 09 23:04:01.909940 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:01.909922 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7106047f-cae5-4ae2-8de3-1b71715187a1-secret-grpc-tls\") pod \"thanos-querier-7f85c99747-bcwgk\" (UID: \"7106047f-cae5-4ae2-8de3-1b71715187a1\") " pod="openshift-monitoring/thanos-querier-7f85c99747-bcwgk" Sep 09 23:04:01.910028 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:01.909966 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/7106047f-cae5-4ae2-8de3-1b71715187a1-secret-thanos-querier-tls\") pod \"thanos-querier-7f85c99747-bcwgk\" (UID: \"7106047f-cae5-4ae2-8de3-1b71715187a1\") " pod="openshift-monitoring/thanos-querier-7f85c99747-bcwgk" Sep 09 23:04:01.910094 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:01.910018 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7106047f-cae5-4ae2-8de3-1b71715187a1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7f85c99747-bcwgk\" (UID: \"7106047f-cae5-4ae2-8de3-1b71715187a1\") " pod="openshift-monitoring/thanos-querier-7f85c99747-bcwgk" Sep 09 23:04:01.914672 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:01.914652 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txp9p\" (UniqueName: \"kubernetes.io/projected/7106047f-cae5-4ae2-8de3-1b71715187a1-kube-api-access-txp9p\") pod \"thanos-querier-7f85c99747-bcwgk\" (UID: \"7106047f-cae5-4ae2-8de3-1b71715187a1\") " pod="openshift-monitoring/thanos-querier-7f85c99747-bcwgk" Sep 09 23:04:02.020900 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:02.020871 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7f85c99747-bcwgk" Sep 09 23:04:02.177634 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:02.177598 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-nw5xf" event={"ID":"92e597ee-8c57-4c77-908e-6e8045ffcded","Type":"ContainerStarted","Data":"96b54a9bdd9874c90f17219b150b309e9cfe8b0d6923d04470a9034447aea805"} Sep 09 23:04:02.179845 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:02.179813 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-gbt9m" event={"ID":"caacaea7-7d91-480b-b93a-e50c9a7f1d25","Type":"ContainerStarted","Data":"91b6dff6580ae97df3e8737227b9f9c152f1c03b7b077c4315862dbdbf61f08a"} Sep 09 23:04:02.179993 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:02.179883 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-gbt9m" event={"ID":"caacaea7-7d91-480b-b93a-e50c9a7f1d25","Type":"ContainerStarted","Data":"d6ae1539f117eaae942b0748bde838d1850d73cd154476d997e0a767a1e2d4d3"} Sep 09 23:04:02.179993 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:02.179916 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-gbt9m" event={"ID":"caacaea7-7d91-480b-b93a-e50c9a7f1d25","Type":"ContainerStarted","Data":"72bd1fe70360f62b3b54e4e9b9445234c17af9a8f82826918b5a488fa2047a2e"} Sep 09 23:04:02.181731 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:02.181693 2580 generic.go:356] "Generic (PLEG): container finished" podID="e2231319-641c-40bb-901b-2577df536ab0" containerID="7b40397c3abd4dd501882e9001062722ba667affad49169539ea1d9376e2042c" exitCode=0 Sep 09 23:04:02.181731 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:02.181730 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wqs6f" event={"ID":"e2231319-641c-40bb-901b-2577df536ab0","Type":"ContainerDied","Data":"7b40397c3abd4dd501882e9001062722ba667affad49169539ea1d9376e2042c"} Sep 09 23:04:02.197095 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:02.196957 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-nw5xf" podStartSLOduration=1.7293094679999998 podStartE2EDuration="3.196937165s" totalImagesPullingTime="1.467627697s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:03:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:04:02.196022416 +0000 UTC m=+303.545889452" watchObservedRunningTime="2026-09-09 23:04:02.196937165 +0000 UTC m=+303.546804201" Sep 09 23:04:02.239528 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:02.239478 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-gbt9m" podStartSLOduration=1.699898314 podStartE2EDuration="3.239460228s" totalImagesPullingTime="1.539561914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:03:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:04:02.238375233 +0000 UTC m=+303.588242302" watchObservedRunningTime="2026-09-09 23:04:02.239460228 +0000 UTC m=+303.589327263" Sep 09 23:04:02.326752 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:02.326724 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7f85c99747-bcwgk"] Sep 09 23:04:02.329382 ip-10-0-131-92 kubenswrapper[2580]: W0909 23:04:02.329344 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7106047f_cae5_4ae2_8de3_1b71715187a1.slice/crio-fc66a16533a9bc70b7601cc7b31df9aa15badd804cde2a249c040e5023b6ce67 WatchSource:0}: Error finding container fc66a16533a9bc70b7601cc7b31df9aa15badd804cde2a249c040e5023b6ce67: Status 404 returned error can't find the container with id fc66a16533a9bc70b7601cc7b31df9aa15badd804cde2a249c040e5023b6ce67 Sep 09 23:04:02.673009 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:02.672957 2580 patch_prober.go:28] interesting pod/image-registry-69855d874b-xvwj9 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 23:04:02.673167 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:02.673017 2580 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-69855d874b-xvwj9" podUID="c15d7152-f7f5-4d4c-8d66-4f955b5b9725" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 23:04:03.192253 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:03.192033 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wqs6f" event={"ID":"e2231319-641c-40bb-901b-2577df536ab0","Type":"ContainerStarted","Data":"34b183b7fb47e830d8c3d9833fee56527cb7010749c6b0255b5240e0528f5fff"} Sep 09 23:04:03.192253 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:03.192092 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wqs6f" event={"ID":"e2231319-641c-40bb-901b-2577df536ab0","Type":"ContainerStarted","Data":"a3b6cd9822041daa48b4fc87e85dbbc1d712c302d386b14bfd692ea47f1356ae"} Sep 09 23:04:03.193912 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:03.193875 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f85c99747-bcwgk" event={"ID":"7106047f-cae5-4ae2-8de3-1b71715187a1","Type":"ContainerStarted","Data":"fc66a16533a9bc70b7601cc7b31df9aa15badd804cde2a249c040e5023b6ce67"} Sep 09 23:04:03.218376 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:03.218314 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-wqs6f" podStartSLOduration=2.336077582 podStartE2EDuration="4.218292967s" totalImagesPullingTime="1.800249494s" totalInitContainerRuntime="81.965891ms" isStatefulPod=true podCreationTimestamp="2026-09-09 23:03:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:04:03.21734847 +0000 UTC m=+304.567215505" watchObservedRunningTime="2026-09-09 23:04:03.218292967 +0000 UTC m=+304.568160020" Sep 09 23:04:04.085469 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:04.085439 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-69855d874b-xvwj9" Sep 09 23:04:05.208641 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:05.208605 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f85c99747-bcwgk" event={"ID":"7106047f-cae5-4ae2-8de3-1b71715187a1","Type":"ContainerStarted","Data":"5222a8b493b23dbd0f852cf8962c90d59077e767ac08f93604dc430a3469755a"} Sep 09 23:04:05.209039 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:05.208696 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f85c99747-bcwgk" event={"ID":"7106047f-cae5-4ae2-8de3-1b71715187a1","Type":"ContainerStarted","Data":"437b447a19a2cbff5439f139d1e7dfaa0fb776236a8395046d0035876dc6c5f6"} Sep 09 23:04:05.209039 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:05.208733 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f85c99747-bcwgk" event={"ID":"7106047f-cae5-4ae2-8de3-1b71715187a1","Type":"ContainerStarted","Data":"52150f29dfd8ceedcf41c3af870f71a5489c15fddcde9564d585b60da48a9691"} Sep 09 23:04:06.020987 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.020890 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 23:04:06.025571 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.025543 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.030366 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.030180 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 09 23:04:06.030366 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.030209 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 09 23:04:06.030366 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.030180 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 09 23:04:06.030366 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.030276 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 09 23:04:06.030366 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.030276 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5jqmv5h73k2hf\"" Sep 09 23:04:06.030366 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.030316 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 09 23:04:06.030768 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.030379 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-92bjv\"" Sep 09 23:04:06.030768 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.030409 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 09 23:04:06.030768 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.030644 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 09 23:04:06.030768 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.030670 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 09 23:04:06.030966 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.030838 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 09 23:04:06.031318 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.031301 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 09 23:04:06.031460 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.031431 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 09 23:04:06.031460 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.031454 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 09 23:04:06.031705 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.031692 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 09 23:04:06.034531 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.034499 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 09 23:04:06.038001 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.037964 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 09 23:04:06.067331 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.067293 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 23:04:06.144747 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.144708 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.144908 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.144762 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.144908 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.144833 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ed2e010-42b2-4939-9a55-f5618b14bb44-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.145134 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.144918 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6ed2e010-42b2-4939-9a55-f5618b14bb44-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.145134 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.144957 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-config\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.145134 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.145035 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-web-config\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.145134 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.145065 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwxng\" (UniqueName: \"kubernetes.io/projected/6ed2e010-42b2-4939-9a55-f5618b14bb44-kube-api-access-qwxng\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.145134 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.145100 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ed2e010-42b2-4939-9a55-f5618b14bb44-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.145134 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.145130 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6ed2e010-42b2-4939-9a55-f5618b14bb44-config-out\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.145352 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.145158 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.145352 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.145206 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6ed2e010-42b2-4939-9a55-f5618b14bb44-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.145352 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.145279 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.145352 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.145325 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6ed2e010-42b2-4939-9a55-f5618b14bb44-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.145516 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.145350 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.145516 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.145375 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6ed2e010-42b2-4939-9a55-f5618b14bb44-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.145516 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.145399 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.145516 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.145441 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6ed2e010-42b2-4939-9a55-f5618b14bb44-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.145516 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.145469 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.145694 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.145512 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6ed2e010-42b2-4939-9a55-f5618b14bb44-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.145694 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.145538 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ed2e010-42b2-4939-9a55-f5618b14bb44-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.173359 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.173331 2580 scope.go:117] "RemoveContainer" containerID="1e344eebcf8cdd0c0c0e7712513dd51097fae6542b42e7a9152c711a218a5f23" Sep 09 23:04:06.173513 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:04:06.173491 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-xht2x_openshift-console-operator(3b6dd849-6a7e-4141-a460-e9ee0e7ff07c)\"" pod="openshift-console-operator/console-operator-85b454874b-xht2x" podUID="3b6dd849-6a7e-4141-a460-e9ee0e7ff07c" Sep 09 23:04:06.216457 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.216421 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f85c99747-bcwgk" event={"ID":"7106047f-cae5-4ae2-8de3-1b71715187a1","Type":"ContainerStarted","Data":"4eeacd601c93e971bf96878b5573ff7cc47e895cea81c191d1603869fbb86969"} Sep 09 23:04:06.217067 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.217040 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f85c99747-bcwgk" event={"ID":"7106047f-cae5-4ae2-8de3-1b71715187a1","Type":"ContainerStarted","Data":"fd2d348218e0e1f71b51a3e64ca91987ce1968417666af1d74c3abd32862bd1e"} Sep 09 23:04:06.217220 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.217205 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f85c99747-bcwgk" event={"ID":"7106047f-cae5-4ae2-8de3-1b71715187a1","Type":"ContainerStarted","Data":"903a32b83cd7d056cedb9e25a39bc80bfd278e2980a99beba9621092e33b4370"} Sep 09 23:04:06.243884 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.243823 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7f85c99747-bcwgk" podStartSLOduration=2.121079634 podStartE2EDuration="5.243799877s" totalImagesPullingTime="3.122720243s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:04:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:04:06.242786702 +0000 UTC m=+307.592653752" watchObservedRunningTime="2026-09-09 23:04:06.243799877 +0000 UTC m=+307.593666913" Sep 09 23:04:06.246261 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.246237 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ed2e010-42b2-4939-9a55-f5618b14bb44-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.246397 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.246268 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6ed2e010-42b2-4939-9a55-f5618b14bb44-config-out\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.246397 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.246285 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.246397 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.246305 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6ed2e010-42b2-4939-9a55-f5618b14bb44-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.246534 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.246513 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.246590 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.246578 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6ed2e010-42b2-4939-9a55-f5618b14bb44-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.246638 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.246609 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.246691 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.246633 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6ed2e010-42b2-4939-9a55-f5618b14bb44-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.246691 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.246658 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.246793 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.246690 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6ed2e010-42b2-4939-9a55-f5618b14bb44-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.246793 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.246715 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.246793 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.246747 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6ed2e010-42b2-4939-9a55-f5618b14bb44-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.246793 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.246773 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ed2e010-42b2-4939-9a55-f5618b14bb44-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.246965 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.246818 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.246965 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.246850 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.246965 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.246894 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ed2e010-42b2-4939-9a55-f5618b14bb44-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.246965 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.246918 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6ed2e010-42b2-4939-9a55-f5618b14bb44-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.246965 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.246924 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6ed2e010-42b2-4939-9a55-f5618b14bb44-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.246965 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.246945 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-config\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.247278 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.247014 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-web-config\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.247278 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.247047 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwxng\" (UniqueName: \"kubernetes.io/projected/6ed2e010-42b2-4939-9a55-f5618b14bb44-kube-api-access-qwxng\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.247278 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.247153 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ed2e010-42b2-4939-9a55-f5618b14bb44-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.247692 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.247659 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6ed2e010-42b2-4939-9a55-f5618b14bb44-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.248531 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.248206 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ed2e010-42b2-4939-9a55-f5618b14bb44-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.250252 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.250226 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6ed2e010-42b2-4939-9a55-f5618b14bb44-config-out\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.250411 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.250390 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6ed2e010-42b2-4939-9a55-f5618b14bb44-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.250621 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.250599 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6ed2e010-42b2-4939-9a55-f5618b14bb44-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.251187 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.251162 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6ed2e010-42b2-4939-9a55-f5618b14bb44-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.251466 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.251438 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.251611 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.251554 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ed2e010-42b2-4939-9a55-f5618b14bb44-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.251730 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.251704 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6ed2e010-42b2-4939-9a55-f5618b14bb44-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.252024 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.251998 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.252122 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.252101 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.254454 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.253869 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.254454 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.254335 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.254825 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.254801 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-web-config\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.254949 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.254907 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.256099 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.256078 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-config\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.256257 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.256237 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.281957 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.281888 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwxng\" (UniqueName: \"kubernetes.io/projected/6ed2e010-42b2-4939-9a55-f5618b14bb44-kube-api-access-qwxng\") pod \"prometheus-k8s-0\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.336913 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.336873 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:06.641578 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:06.641542 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 23:04:06.644432 ip-10-0-131-92 kubenswrapper[2580]: W0909 23:04:06.644404 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ed2e010_42b2_4939_9a55_f5618b14bb44.slice/crio-57711456233a24e971674397953e7fbb2260ceca32df3da81ff6fdc9923da957 WatchSource:0}: Error finding container 57711456233a24e971674397953e7fbb2260ceca32df3da81ff6fdc9923da957: Status 404 returned error can't find the container with id 57711456233a24e971674397953e7fbb2260ceca32df3da81ff6fdc9923da957 Sep 09 23:04:07.179552 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:07.179486 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dwkd" Sep 09 23:04:07.224063 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:07.223952 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6ed2e010-42b2-4939-9a55-f5618b14bb44","Type":"ContainerStarted","Data":"57711456233a24e971674397953e7fbb2260ceca32df3da81ff6fdc9923da957"} Sep 09 23:04:07.224533 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:07.224334 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7f85c99747-bcwgk" Sep 09 23:04:07.782762 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:07.782726 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-69855d874b-xvwj9"] Sep 09 23:04:08.231406 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:08.231365 2580 generic.go:356] "Generic (PLEG): container finished" podID="6ed2e010-42b2-4939-9a55-f5618b14bb44" containerID="2c8f39e124a92569d5ed80c9e35e532d138dd2fe22edc379f6f12db6e72eeb6c" exitCode=0 Sep 09 23:04:08.231846 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:08.231456 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6ed2e010-42b2-4939-9a55-f5618b14bb44","Type":"ContainerDied","Data":"2c8f39e124a92569d5ed80c9e35e532d138dd2fe22edc379f6f12db6e72eeb6c"} Sep 09 23:04:12.254740 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:12.254702 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6ed2e010-42b2-4939-9a55-f5618b14bb44","Type":"ContainerStarted","Data":"8555484ea30c02030b2e41d62fd232163c16c465f336ba7712045365afaba797"} Sep 09 23:04:12.255212 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:12.254765 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6ed2e010-42b2-4939-9a55-f5618b14bb44","Type":"ContainerStarted","Data":"6a2e88830a67214c3d42cd452e650d030cdbe4d443b296db90d58c68c98eae35"} Sep 09 23:04:12.255212 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:12.254790 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6ed2e010-42b2-4939-9a55-f5618b14bb44","Type":"ContainerStarted","Data":"e144d770caa8b7ac6bbbc963afa93c0c2e11791941a7ec9efb9970c707aca5e3"} Sep 09 23:04:12.255212 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:12.254814 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6ed2e010-42b2-4939-9a55-f5618b14bb44","Type":"ContainerStarted","Data":"aae4861c6d8159a840a3584b049461ad699f6e9283cff1fa037f9974f4cf02a6"} Sep 09 23:04:12.255212 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:12.254842 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6ed2e010-42b2-4939-9a55-f5618b14bb44","Type":"ContainerStarted","Data":"2451c3bccea5fbab5d92cde891868e89627fbf9ffd086a3692a44a4f956abef6"} Sep 09 23:04:12.255212 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:12.254869 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6ed2e010-42b2-4939-9a55-f5618b14bb44","Type":"ContainerStarted","Data":"469ff6f120894e4caa0d649903baa4b9f12c46bc1bf596e808196545e4175e3c"} Sep 09 23:04:12.288608 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:12.288555 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.104007157 podStartE2EDuration="7.28853922s" totalImagesPullingTime="4.165982518s" totalInitContainerRuntime="18.549545ms" isStatefulPod=false podCreationTimestamp="2026-09-09 23:04:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:04:12.285674134 +0000 UTC m=+313.635541168" watchObservedRunningTime="2026-09-09 23:04:12.28853922 +0000 UTC m=+313.638406254" Sep 09 23:04:13.237889 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:13.237859 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7f85c99747-bcwgk" Sep 09 23:04:16.337991 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:16.337934 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:04:18.173295 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:18.173265 2580 scope.go:117] "RemoveContainer" containerID="1e344eebcf8cdd0c0c0e7712513dd51097fae6542b42e7a9152c711a218a5f23" Sep 09 23:04:18.282090 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:18.282063 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xht2x_3b6dd849-6a7e-4141-a460-e9ee0e7ff07c/console-operator/2.log" Sep 09 23:04:18.282238 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:18.282182 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-xht2x" event={"ID":"3b6dd849-6a7e-4141-a460-e9ee0e7ff07c","Type":"ContainerStarted","Data":"873e89f0fcbc45774aac3377aa0af5b87f46aed04b30b72cf8ef8306098a8a1f"} Sep 09 23:04:18.282491 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:18.282466 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-xht2x" Sep 09 23:04:18.283097 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:18.283074 2580 patch_prober.go:28] interesting pod/console-operator-85b454874b-xht2x container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.12:8443/readyz\": dial tcp 10.132.0.12:8443: connect: connection refused" start-of-body= Sep 09 23:04:18.283212 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:18.283132 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-85b454874b-xht2x" podUID="3b6dd849-6a7e-4141-a460-e9ee0e7ff07c" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.12:8443/readyz\": dial tcp 10.132.0.12:8443: connect: connection refused" Sep 09 23:04:19.289708 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:19.289680 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-xht2x" Sep 09 23:04:28.322606 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:28.322530 2580 generic.go:356] "Generic (PLEG): container finished" podID="4ebd018d-e004-4bd2-94ed-594b76e747ca" containerID="931ab674255655e97aa7d917eee3c9587333ee3f1073c7fa449f3648ee2031cf" exitCode=0 Sep 09 23:04:28.322936 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:28.322603 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dtbbq" event={"ID":"4ebd018d-e004-4bd2-94ed-594b76e747ca","Type":"ContainerDied","Data":"931ab674255655e97aa7d917eee3c9587333ee3f1073c7fa449f3648ee2031cf"} Sep 09 23:04:28.323011 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:28.322947 2580 scope.go:117] "RemoveContainer" containerID="931ab674255655e97aa7d917eee3c9587333ee3f1073c7fa449f3648ee2031cf" Sep 09 23:04:29.333961 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:29.333926 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dtbbq" event={"ID":"4ebd018d-e004-4bd2-94ed-594b76e747ca","Type":"ContainerStarted","Data":"1a7b9de5892ef61e6702a5df4225c54a90d7dd508b70d9dbf8211391adebf361"} Sep 09 23:04:32.802047 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:32.802000 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-69855d874b-xvwj9" podUID="c15d7152-f7f5-4d4c-8d66-4f955b5b9725" containerName="registry" containerID="cri-o://24614d928ae765956c53ae21a08e1a815c436b81d60c03bf70ffca4a57d42b97" gracePeriod=30 Sep 09 23:04:33.054219 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:33.054160 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69855d874b-xvwj9" Sep 09 23:04:33.085369 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:33.085329 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-installation-pull-secrets\") pod \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\" (UID: \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\") " Sep 09 23:04:33.085531 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:33.085411 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-image-registry-private-configuration\") pod \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\" (UID: \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\") " Sep 09 23:04:33.085531 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:33.085441 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-ca-trust-extracted\") pod \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\" (UID: \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\") " Sep 09 23:04:33.085531 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:33.085467 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-registry-tls\") pod \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\" (UID: \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\") " Sep 09 23:04:33.085531 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:33.085496 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-registry-certificates\") pod \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\" (UID: \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\") " Sep 09 23:04:33.085739 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:33.085553 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-bound-sa-token\") pod \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\" (UID: \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\") " Sep 09 23:04:33.085739 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:33.085603 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-289cb\" (UniqueName: \"kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-kube-api-access-289cb\") pod \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\" (UID: \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\") " Sep 09 23:04:33.085739 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:33.085630 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-trusted-ca\") pod \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\" (UID: \"c15d7152-f7f5-4d4c-8d66-4f955b5b9725\") " Sep 09 23:04:33.086343 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:33.086313 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "c15d7152-f7f5-4d4c-8d66-4f955b5b9725" (UID: "c15d7152-f7f5-4d4c-8d66-4f955b5b9725"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 23:04:33.086613 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:33.086578 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "c15d7152-f7f5-4d4c-8d66-4f955b5b9725" (UID: "c15d7152-f7f5-4d4c-8d66-4f955b5b9725"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 23:04:33.088281 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:33.088224 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "c15d7152-f7f5-4d4c-8d66-4f955b5b9725" (UID: "c15d7152-f7f5-4d4c-8d66-4f955b5b9725"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 23:04:33.088369 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:33.088290 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "c15d7152-f7f5-4d4c-8d66-4f955b5b9725" (UID: "c15d7152-f7f5-4d4c-8d66-4f955b5b9725"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 23:04:33.088447 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:33.088426 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "c15d7152-f7f5-4d4c-8d66-4f955b5b9725" (UID: "c15d7152-f7f5-4d4c-8d66-4f955b5b9725"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 23:04:33.088689 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:33.088646 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-kube-api-access-289cb" (OuterVolumeSpecName: "kube-api-access-289cb") pod "c15d7152-f7f5-4d4c-8d66-4f955b5b9725" (UID: "c15d7152-f7f5-4d4c-8d66-4f955b5b9725"). InnerVolumeSpecName "kube-api-access-289cb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 23:04:33.088886 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:33.088871 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c15d7152-f7f5-4d4c-8d66-4f955b5b9725" (UID: "c15d7152-f7f5-4d4c-8d66-4f955b5b9725"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 23:04:33.095637 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:33.095613 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "c15d7152-f7f5-4d4c-8d66-4f955b5b9725" (UID: "c15d7152-f7f5-4d4c-8d66-4f955b5b9725"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 23:04:33.187085 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:33.187060 2580 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-image-registry-private-configuration\") on node \"ip-10-0-131-92.ec2.internal\" DevicePath \"\"" Sep 09 23:04:33.187085 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:33.187084 2580 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-ca-trust-extracted\") on node \"ip-10-0-131-92.ec2.internal\" DevicePath \"\"" Sep 09 23:04:33.187227 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:33.187095 2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-registry-tls\") on node \"ip-10-0-131-92.ec2.internal\" DevicePath \"\"" Sep 09 23:04:33.187227 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:33.187103 2580 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-registry-certificates\") on node \"ip-10-0-131-92.ec2.internal\" DevicePath \"\"" Sep 09 23:04:33.187227 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:33.187111 2580 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-bound-sa-token\") on node \"ip-10-0-131-92.ec2.internal\" DevicePath \"\"" Sep 09 23:04:33.187227 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:33.187120 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-289cb\" (UniqueName: \"kubernetes.io/projected/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-kube-api-access-289cb\") on node \"ip-10-0-131-92.ec2.internal\" DevicePath \"\"" Sep 09 23:04:33.187227 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:33.187127 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-trusted-ca\") on node \"ip-10-0-131-92.ec2.internal\" DevicePath \"\"" Sep 09 23:04:33.187227 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:33.187137 2580 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c15d7152-f7f5-4d4c-8d66-4f955b5b9725-installation-pull-secrets\") on node \"ip-10-0-131-92.ec2.internal\" DevicePath \"\"" Sep 09 23:04:33.350535 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:33.350442 2580 generic.go:356] "Generic (PLEG): container finished" podID="c15d7152-f7f5-4d4c-8d66-4f955b5b9725" containerID="24614d928ae765956c53ae21a08e1a815c436b81d60c03bf70ffca4a57d42b97" exitCode=0 Sep 09 23:04:33.350535 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:33.350485 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69855d874b-xvwj9" Sep 09 23:04:33.350535 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:33.350528 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69855d874b-xvwj9" event={"ID":"c15d7152-f7f5-4d4c-8d66-4f955b5b9725","Type":"ContainerDied","Data":"24614d928ae765956c53ae21a08e1a815c436b81d60c03bf70ffca4a57d42b97"} Sep 09 23:04:33.350797 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:33.350575 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69855d874b-xvwj9" event={"ID":"c15d7152-f7f5-4d4c-8d66-4f955b5b9725","Type":"ContainerDied","Data":"217972ed9b37df73b542ceb7d6374950a0a40e87d9d1d0c2d4ed232751b70034"} Sep 09 23:04:33.350797 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:33.350598 2580 scope.go:117] "RemoveContainer" containerID="24614d928ae765956c53ae21a08e1a815c436b81d60c03bf70ffca4a57d42b97" Sep 09 23:04:33.358509 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:33.358490 2580 scope.go:117] "RemoveContainer" containerID="24614d928ae765956c53ae21a08e1a815c436b81d60c03bf70ffca4a57d42b97" Sep 09 23:04:33.358787 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:04:33.358763 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24614d928ae765956c53ae21a08e1a815c436b81d60c03bf70ffca4a57d42b97\": container with ID starting with 24614d928ae765956c53ae21a08e1a815c436b81d60c03bf70ffca4a57d42b97 not found: ID does not exist" containerID="24614d928ae765956c53ae21a08e1a815c436b81d60c03bf70ffca4a57d42b97" Sep 09 23:04:33.358857 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:33.358800 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24614d928ae765956c53ae21a08e1a815c436b81d60c03bf70ffca4a57d42b97"} err="failed to get container status \"24614d928ae765956c53ae21a08e1a815c436b81d60c03bf70ffca4a57d42b97\": rpc error: code = NotFound desc = could not find container \"24614d928ae765956c53ae21a08e1a815c436b81d60c03bf70ffca4a57d42b97\": container with ID starting with 24614d928ae765956c53ae21a08e1a815c436b81d60c03bf70ffca4a57d42b97 not found: ID does not exist" Sep 09 23:04:33.367151 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:33.367127 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-69855d874b-xvwj9"] Sep 09 23:04:33.370923 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:33.370901 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-69855d874b-xvwj9"] Sep 09 23:04:35.178886 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:35.178848 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c15d7152-f7f5-4d4c-8d66-4f955b5b9725" path="/var/lib/kubelet/pods/c15d7152-f7f5-4d4c-8d66-4f955b5b9725/volumes" Sep 09 23:04:43.392963 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:43.392922 2580 generic.go:356] "Generic (PLEG): container finished" podID="00cae89a-a6b7-401f-8fde-93e5f33890da" containerID="05751b8cad31a9822128f9421a8f098b769566ef65cf5b225c2947cae1365ccc" exitCode=0 Sep 09 23:04:43.393539 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:43.393011 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qctkt" event={"ID":"00cae89a-a6b7-401f-8fde-93e5f33890da","Type":"ContainerDied","Data":"05751b8cad31a9822128f9421a8f098b769566ef65cf5b225c2947cae1365ccc"} Sep 09 23:04:43.393539 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:43.393384 2580 scope.go:117] "RemoveContainer" containerID="05751b8cad31a9822128f9421a8f098b769566ef65cf5b225c2947cae1365ccc" Sep 09 23:04:44.400027 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:04:44.399990 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qctkt" event={"ID":"00cae89a-a6b7-401f-8fde-93e5f33890da","Type":"ContainerStarted","Data":"482cb55d121c57a384ac1986aa4f740ca65d8efbb0b5d6005567d8ddc72b8dae"} Sep 09 23:05:06.337541 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:06.337500 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:06.353192 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:06.353167 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:06.499595 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:06.499570 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:13.041293 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:13.041249 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cbe663c-0531-4339-b48b-c4d4c3dd5c2b-metrics-certs\") pod \"network-metrics-daemon-8dwkd\" (UID: \"7cbe663c-0531-4339-b48b-c4d4c3dd5c2b\") " pod="openshift-multus/network-metrics-daemon-8dwkd" Sep 09 23:05:13.043524 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:13.043494 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cbe663c-0531-4339-b48b-c4d4c3dd5c2b-metrics-certs\") pod \"network-metrics-daemon-8dwkd\" (UID: \"7cbe663c-0531-4339-b48b-c4d4c3dd5c2b\") " pod="openshift-multus/network-metrics-daemon-8dwkd" Sep 09 23:05:13.187573 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:13.183383 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vjfnz\"" Sep 09 23:05:13.190423 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:13.190401 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dwkd" Sep 09 23:05:13.459766 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:13.459578 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8dwkd"] Sep 09 23:05:13.462491 ip-10-0-131-92 kubenswrapper[2580]: W0909 23:05:13.462462 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cbe663c_0531_4339_b48b_c4d4c3dd5c2b.slice/crio-528eb730d626ce200e07f011f904bcd20af939e9d8a31a5069379d1d9ccf624e WatchSource:0}: Error finding container 528eb730d626ce200e07f011f904bcd20af939e9d8a31a5069379d1d9ccf624e: Status 404 returned error can't find the container with id 528eb730d626ce200e07f011f904bcd20af939e9d8a31a5069379d1d9ccf624e Sep 09 23:05:13.510951 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:13.510913 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8dwkd" event={"ID":"7cbe663c-0531-4339-b48b-c4d4c3dd5c2b","Type":"ContainerStarted","Data":"528eb730d626ce200e07f011f904bcd20af939e9d8a31a5069379d1d9ccf624e"} Sep 09 23:05:15.521805 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:15.521767 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8dwkd" event={"ID":"7cbe663c-0531-4339-b48b-c4d4c3dd5c2b","Type":"ContainerStarted","Data":"a4e73e5c1d445d6dbc91acc4e795965a6c1725eb92666df078bb209f0be4699f"} Sep 09 23:05:15.522258 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:15.521822 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8dwkd" event={"ID":"7cbe663c-0531-4339-b48b-c4d4c3dd5c2b","Type":"ContainerStarted","Data":"1d61cfb832c3db77cd9d773d68bbb05867efe71f01ae049ad92459458c652cff"} Sep 09 23:05:15.539028 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:15.538965 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-8dwkd" podStartSLOduration=375.41253667 podStartE2EDuration="6m16.538949116s" totalImagesPullingTime="1.126412446s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 22:58:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:05:15.536838485 +0000 UTC m=+376.886705539" watchObservedRunningTime="2026-09-09 23:05:15.538949116 +0000 UTC m=+376.888816149" Sep 09 23:05:24.365534 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.365492 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 23:05:24.366517 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.366477 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="6ed2e010-42b2-4939-9a55-f5618b14bb44" containerName="prometheus" containerID="cri-o://469ff6f120894e4caa0d649903baa4b9f12c46bc1bf596e808196545e4175e3c" gracePeriod=600 Sep 09 23:05:24.366670 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.366570 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="6ed2e010-42b2-4939-9a55-f5618b14bb44" containerName="kube-rbac-proxy-web" containerID="cri-o://e144d770caa8b7ac6bbbc963afa93c0c2e11791941a7ec9efb9970c707aca5e3" gracePeriod=600 Sep 09 23:05:24.366670 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.366615 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="6ed2e010-42b2-4939-9a55-f5618b14bb44" containerName="kube-rbac-proxy-thanos" containerID="cri-o://8555484ea30c02030b2e41d62fd232163c16c465f336ba7712045365afaba797" gracePeriod=600 Sep 09 23:05:24.366796 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.366679 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="6ed2e010-42b2-4939-9a55-f5618b14bb44" containerName="thanos-sidecar" containerID="cri-o://aae4861c6d8159a840a3584b049461ad699f6e9283cff1fa037f9974f4cf02a6" gracePeriod=600 Sep 09 23:05:24.366882 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.366859 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="6ed2e010-42b2-4939-9a55-f5618b14bb44" containerName="config-reloader" containerID="cri-o://2451c3bccea5fbab5d92cde891868e89627fbf9ffd086a3692a44a4f956abef6" gracePeriod=600 Sep 09 23:05:24.369991 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.367213 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="6ed2e010-42b2-4939-9a55-f5618b14bb44" containerName="kube-rbac-proxy" containerID="cri-o://6a2e88830a67214c3d42cd452e650d030cdbe4d443b296db90d58c68c98eae35" gracePeriod=600 Sep 09 23:05:24.566559 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.566520 2580 generic.go:356] "Generic (PLEG): container finished" podID="6ed2e010-42b2-4939-9a55-f5618b14bb44" containerID="8555484ea30c02030b2e41d62fd232163c16c465f336ba7712045365afaba797" exitCode=0 Sep 09 23:05:24.566559 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.566556 2580 generic.go:356] "Generic (PLEG): container finished" podID="6ed2e010-42b2-4939-9a55-f5618b14bb44" containerID="6a2e88830a67214c3d42cd452e650d030cdbe4d443b296db90d58c68c98eae35" exitCode=0 Sep 09 23:05:24.566559 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.566566 2580 generic.go:356] "Generic (PLEG): container finished" podID="6ed2e010-42b2-4939-9a55-f5618b14bb44" containerID="e144d770caa8b7ac6bbbc963afa93c0c2e11791941a7ec9efb9970c707aca5e3" exitCode=0 Sep 09 23:05:24.566559 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.566574 2580 generic.go:356] "Generic (PLEG): container finished" podID="6ed2e010-42b2-4939-9a55-f5618b14bb44" containerID="aae4861c6d8159a840a3584b049461ad699f6e9283cff1fa037f9974f4cf02a6" exitCode=0 Sep 09 23:05:24.566902 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.566583 2580 generic.go:356] "Generic (PLEG): container finished" podID="6ed2e010-42b2-4939-9a55-f5618b14bb44" containerID="2451c3bccea5fbab5d92cde891868e89627fbf9ffd086a3692a44a4f956abef6" exitCode=0 Sep 09 23:05:24.566902 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.566591 2580 generic.go:356] "Generic (PLEG): container finished" podID="6ed2e010-42b2-4939-9a55-f5618b14bb44" containerID="469ff6f120894e4caa0d649903baa4b9f12c46bc1bf596e808196545e4175e3c" exitCode=0 Sep 09 23:05:24.566902 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.566649 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6ed2e010-42b2-4939-9a55-f5618b14bb44","Type":"ContainerDied","Data":"8555484ea30c02030b2e41d62fd232163c16c465f336ba7712045365afaba797"} Sep 09 23:05:24.566902 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.566755 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6ed2e010-42b2-4939-9a55-f5618b14bb44","Type":"ContainerDied","Data":"6a2e88830a67214c3d42cd452e650d030cdbe4d443b296db90d58c68c98eae35"} Sep 09 23:05:24.566902 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.566794 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6ed2e010-42b2-4939-9a55-f5618b14bb44","Type":"ContainerDied","Data":"e144d770caa8b7ac6bbbc963afa93c0c2e11791941a7ec9efb9970c707aca5e3"} Sep 09 23:05:24.566902 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.566833 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6ed2e010-42b2-4939-9a55-f5618b14bb44","Type":"ContainerDied","Data":"aae4861c6d8159a840a3584b049461ad699f6e9283cff1fa037f9974f4cf02a6"} Sep 09 23:05:24.566902 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.566873 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6ed2e010-42b2-4939-9a55-f5618b14bb44","Type":"ContainerDied","Data":"2451c3bccea5fbab5d92cde891868e89627fbf9ffd086a3692a44a4f956abef6"} Sep 09 23:05:24.567338 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.566913 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6ed2e010-42b2-4939-9a55-f5618b14bb44","Type":"ContainerDied","Data":"469ff6f120894e4caa0d649903baa4b9f12c46bc1bf596e808196545e4175e3c"} Sep 09 23:05:24.622748 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.622723 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:24.732739 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.732707 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6ed2e010-42b2-4939-9a55-f5618b14bb44-config-out\") pod \"6ed2e010-42b2-4939-9a55-f5618b14bb44\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " Sep 09 23:05:24.732910 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.732778 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6ed2e010-42b2-4939-9a55-f5618b14bb44-prometheus-k8s-db\") pod \"6ed2e010-42b2-4939-9a55-f5618b14bb44\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " Sep 09 23:05:24.732910 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.732805 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-secret-kube-rbac-proxy\") pod \"6ed2e010-42b2-4939-9a55-f5618b14bb44\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " Sep 09 23:05:24.732910 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.732824 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-web-config\") pod \"6ed2e010-42b2-4939-9a55-f5618b14bb44\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " Sep 09 23:05:24.732910 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.732844 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"6ed2e010-42b2-4939-9a55-f5618b14bb44\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " Sep 09 23:05:24.732910 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.732871 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-config\") pod \"6ed2e010-42b2-4939-9a55-f5618b14bb44\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " Sep 09 23:05:24.732910 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.732896 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6ed2e010-42b2-4939-9a55-f5618b14bb44-prometheus-k8s-rulefiles-2\") pod \"6ed2e010-42b2-4939-9a55-f5618b14bb44\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " Sep 09 23:05:24.733232 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.732918 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-secret-metrics-client-certs\") pod \"6ed2e010-42b2-4939-9a55-f5618b14bb44\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " Sep 09 23:05:24.733232 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.732942 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6ed2e010-42b2-4939-9a55-f5618b14bb44-prometheus-k8s-rulefiles-1\") pod \"6ed2e010-42b2-4939-9a55-f5618b14bb44\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " Sep 09 23:05:24.733232 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.732983 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-secret-prometheus-k8s-tls\") pod \"6ed2e010-42b2-4939-9a55-f5618b14bb44\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " Sep 09 23:05:24.733386 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.733334 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ed2e010-42b2-4939-9a55-f5618b14bb44-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "6ed2e010-42b2-4939-9a55-f5618b14bb44" (UID: "6ed2e010-42b2-4939-9a55-f5618b14bb44"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 23:05:24.733446 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.733400 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"6ed2e010-42b2-4939-9a55-f5618b14bb44\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " Sep 09 23:05:24.733503 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.733450 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ed2e010-42b2-4939-9a55-f5618b14bb44-configmap-kubelet-serving-ca-bundle\") pod \"6ed2e010-42b2-4939-9a55-f5618b14bb44\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " Sep 09 23:05:24.733503 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.733482 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ed2e010-42b2-4939-9a55-f5618b14bb44-configmap-serving-certs-ca-bundle\") pod \"6ed2e010-42b2-4939-9a55-f5618b14bb44\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " Sep 09 23:05:24.733618 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.733506 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6ed2e010-42b2-4939-9a55-f5618b14bb44-prometheus-k8s-rulefiles-0\") pod \"6ed2e010-42b2-4939-9a55-f5618b14bb44\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " Sep 09 23:05:24.733618 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.733534 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6ed2e010-42b2-4939-9a55-f5618b14bb44-tls-assets\") pod \"6ed2e010-42b2-4939-9a55-f5618b14bb44\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " Sep 09 23:05:24.733618 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.733566 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-thanos-prometheus-http-client-file\") pod \"6ed2e010-42b2-4939-9a55-f5618b14bb44\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " Sep 09 23:05:24.733618 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.733599 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ed2e010-42b2-4939-9a55-f5618b14bb44-prometheus-trusted-ca-bundle\") pod \"6ed2e010-42b2-4939-9a55-f5618b14bb44\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " Sep 09 23:05:24.733828 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.733631 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6ed2e010-42b2-4939-9a55-f5618b14bb44-configmap-metrics-client-ca\") pod \"6ed2e010-42b2-4939-9a55-f5618b14bb44\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " Sep 09 23:05:24.733828 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.733662 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qwxng\" (UniqueName: \"kubernetes.io/projected/6ed2e010-42b2-4939-9a55-f5618b14bb44-kube-api-access-qwxng\") pod \"6ed2e010-42b2-4939-9a55-f5618b14bb44\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " Sep 09 23:05:24.733828 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.733700 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-secret-grpc-tls\") pod \"6ed2e010-42b2-4939-9a55-f5618b14bb44\" (UID: \"6ed2e010-42b2-4939-9a55-f5618b14bb44\") " Sep 09 23:05:24.734002 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.733892 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ed2e010-42b2-4939-9a55-f5618b14bb44-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "6ed2e010-42b2-4939-9a55-f5618b14bb44" (UID: "6ed2e010-42b2-4939-9a55-f5618b14bb44"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 23:05:24.734064 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.734039 2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6ed2e010-42b2-4939-9a55-f5618b14bb44-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-131-92.ec2.internal\" DevicePath \"\"" Sep 09 23:05:24.734133 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.734061 2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6ed2e010-42b2-4939-9a55-f5618b14bb44-prometheus-k8s-db\") on node \"ip-10-0-131-92.ec2.internal\" DevicePath \"\"" Sep 09 23:05:24.734899 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.734560 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ed2e010-42b2-4939-9a55-f5618b14bb44-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "6ed2e010-42b2-4939-9a55-f5618b14bb44" (UID: "6ed2e010-42b2-4939-9a55-f5618b14bb44"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 23:05:24.734899 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.734787 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ed2e010-42b2-4939-9a55-f5618b14bb44-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "6ed2e010-42b2-4939-9a55-f5618b14bb44" (UID: "6ed2e010-42b2-4939-9a55-f5618b14bb44"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 23:05:24.735277 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.735243 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ed2e010-42b2-4939-9a55-f5618b14bb44-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "6ed2e010-42b2-4939-9a55-f5618b14bb44" (UID: "6ed2e010-42b2-4939-9a55-f5618b14bb44"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 23:05:24.735694 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.735661 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ed2e010-42b2-4939-9a55-f5618b14bb44-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "6ed2e010-42b2-4939-9a55-f5618b14bb44" (UID: "6ed2e010-42b2-4939-9a55-f5618b14bb44"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 23:05:24.735694 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.735679 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ed2e010-42b2-4939-9a55-f5618b14bb44-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "6ed2e010-42b2-4939-9a55-f5618b14bb44" (UID: "6ed2e010-42b2-4939-9a55-f5618b14bb44"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 23:05:24.735834 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.735753 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "6ed2e010-42b2-4939-9a55-f5618b14bb44" (UID: "6ed2e010-42b2-4939-9a55-f5618b14bb44"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 23:05:24.736228 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.736184 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ed2e010-42b2-4939-9a55-f5618b14bb44-config-out" (OuterVolumeSpecName: "config-out") pod "6ed2e010-42b2-4939-9a55-f5618b14bb44" (UID: "6ed2e010-42b2-4939-9a55-f5618b14bb44"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 23:05:24.736559 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.736522 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "6ed2e010-42b2-4939-9a55-f5618b14bb44" (UID: "6ed2e010-42b2-4939-9a55-f5618b14bb44"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 23:05:24.736700 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.736563 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "6ed2e010-42b2-4939-9a55-f5618b14bb44" (UID: "6ed2e010-42b2-4939-9a55-f5618b14bb44"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 23:05:24.736923 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.736896 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ed2e010-42b2-4939-9a55-f5618b14bb44-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "6ed2e010-42b2-4939-9a55-f5618b14bb44" (UID: "6ed2e010-42b2-4939-9a55-f5618b14bb44"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 23:05:24.737185 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.737158 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ed2e010-42b2-4939-9a55-f5618b14bb44-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "6ed2e010-42b2-4939-9a55-f5618b14bb44" (UID: "6ed2e010-42b2-4939-9a55-f5618b14bb44"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 23:05:24.737847 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.737814 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "6ed2e010-42b2-4939-9a55-f5618b14bb44" (UID: "6ed2e010-42b2-4939-9a55-f5618b14bb44"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 23:05:24.737939 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.737844 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "6ed2e010-42b2-4939-9a55-f5618b14bb44" (UID: "6ed2e010-42b2-4939-9a55-f5618b14bb44"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 23:05:24.738123 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.738102 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ed2e010-42b2-4939-9a55-f5618b14bb44-kube-api-access-qwxng" (OuterVolumeSpecName: "kube-api-access-qwxng") pod "6ed2e010-42b2-4939-9a55-f5618b14bb44" (UID: "6ed2e010-42b2-4939-9a55-f5618b14bb44"). InnerVolumeSpecName "kube-api-access-qwxng". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 23:05:24.738204 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.738185 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "6ed2e010-42b2-4939-9a55-f5618b14bb44" (UID: "6ed2e010-42b2-4939-9a55-f5618b14bb44"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 23:05:24.738352 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.738326 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "6ed2e010-42b2-4939-9a55-f5618b14bb44" (UID: "6ed2e010-42b2-4939-9a55-f5618b14bb44"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 23:05:24.738561 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.738548 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-config" (OuterVolumeSpecName: "config") pod "6ed2e010-42b2-4939-9a55-f5618b14bb44" (UID: "6ed2e010-42b2-4939-9a55-f5618b14bb44"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 23:05:24.748264 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.748244 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-web-config" (OuterVolumeSpecName: "web-config") pod "6ed2e010-42b2-4939-9a55-f5618b14bb44" (UID: "6ed2e010-42b2-4939-9a55-f5618b14bb44"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 23:05:24.834813 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.834770 2580 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-secret-kube-rbac-proxy\") on node \"ip-10-0-131-92.ec2.internal\" DevicePath \"\"" Sep 09 23:05:24.834813 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.834804 2580 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-web-config\") on node \"ip-10-0-131-92.ec2.internal\" DevicePath \"\"" Sep 09 23:05:24.834813 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.834817 2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-131-92.ec2.internal\" DevicePath \"\"" Sep 09 23:05:24.834813 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.834826 2580 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-config\") on node \"ip-10-0-131-92.ec2.internal\" DevicePath \"\"" Sep 09 23:05:24.835113 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.834835 2580 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-secret-metrics-client-certs\") on node \"ip-10-0-131-92.ec2.internal\" DevicePath \"\"" Sep 09 23:05:24.835113 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.834843 2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6ed2e010-42b2-4939-9a55-f5618b14bb44-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-131-92.ec2.internal\" DevicePath \"\"" Sep 09 23:05:24.835113 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.834853 2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-secret-prometheus-k8s-tls\") on node \"ip-10-0-131-92.ec2.internal\" DevicePath \"\"" Sep 09 23:05:24.835113 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.834861 2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-131-92.ec2.internal\" DevicePath \"\"" Sep 09 23:05:24.835113 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.834870 2580 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ed2e010-42b2-4939-9a55-f5618b14bb44-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-131-92.ec2.internal\" DevicePath \"\"" Sep 09 23:05:24.835113 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.834878 2580 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ed2e010-42b2-4939-9a55-f5618b14bb44-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-131-92.ec2.internal\" DevicePath \"\"" Sep 09 23:05:24.835113 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.834886 2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6ed2e010-42b2-4939-9a55-f5618b14bb44-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-131-92.ec2.internal\" DevicePath \"\"" Sep 09 23:05:24.835113 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.834894 2580 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6ed2e010-42b2-4939-9a55-f5618b14bb44-tls-assets\") on node \"ip-10-0-131-92.ec2.internal\" DevicePath \"\"" Sep 09 23:05:24.835113 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.834901 2580 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-thanos-prometheus-http-client-file\") on node \"ip-10-0-131-92.ec2.internal\" DevicePath \"\"" Sep 09 23:05:24.835113 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.834909 2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ed2e010-42b2-4939-9a55-f5618b14bb44-prometheus-trusted-ca-bundle\") on node \"ip-10-0-131-92.ec2.internal\" DevicePath \"\"" Sep 09 23:05:24.835113 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.834917 2580 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6ed2e010-42b2-4939-9a55-f5618b14bb44-configmap-metrics-client-ca\") on node \"ip-10-0-131-92.ec2.internal\" DevicePath \"\"" Sep 09 23:05:24.835113 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.834925 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qwxng\" (UniqueName: \"kubernetes.io/projected/6ed2e010-42b2-4939-9a55-f5618b14bb44-kube-api-access-qwxng\") on node \"ip-10-0-131-92.ec2.internal\" DevicePath \"\"" Sep 09 23:05:24.835113 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.834935 2580 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6ed2e010-42b2-4939-9a55-f5618b14bb44-secret-grpc-tls\") on node \"ip-10-0-131-92.ec2.internal\" DevicePath \"\"" Sep 09 23:05:24.835113 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:24.834943 2580 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6ed2e010-42b2-4939-9a55-f5618b14bb44-config-out\") on node \"ip-10-0-131-92.ec2.internal\" DevicePath \"\"" Sep 09 23:05:25.573667 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.573591 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6ed2e010-42b2-4939-9a55-f5618b14bb44","Type":"ContainerDied","Data":"57711456233a24e971674397953e7fbb2260ceca32df3da81ff6fdc9923da957"} Sep 09 23:05:25.573667 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.573610 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.573667 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.573665 2580 scope.go:117] "RemoveContainer" containerID="8555484ea30c02030b2e41d62fd232163c16c465f336ba7712045365afaba797" Sep 09 23:05:25.581379 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.581362 2580 scope.go:117] "RemoveContainer" containerID="6a2e88830a67214c3d42cd452e650d030cdbe4d443b296db90d58c68c98eae35" Sep 09 23:05:25.588446 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.588429 2580 scope.go:117] "RemoveContainer" containerID="e144d770caa8b7ac6bbbc963afa93c0c2e11791941a7ec9efb9970c707aca5e3" Sep 09 23:05:25.595204 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.595183 2580 scope.go:117] "RemoveContainer" containerID="aae4861c6d8159a840a3584b049461ad699f6e9283cff1fa037f9974f4cf02a6" Sep 09 23:05:25.597594 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.597569 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 23:05:25.601414 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.601392 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 23:05:25.602658 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.602641 2580 scope.go:117] "RemoveContainer" containerID="2451c3bccea5fbab5d92cde891868e89627fbf9ffd086a3692a44a4f956abef6" Sep 09 23:05:25.609150 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.609133 2580 scope.go:117] "RemoveContainer" containerID="469ff6f120894e4caa0d649903baa4b9f12c46bc1bf596e808196545e4175e3c" Sep 09 23:05:25.616119 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.616097 2580 scope.go:117] "RemoveContainer" containerID="2c8f39e124a92569d5ed80c9e35e532d138dd2fe22edc379f6f12db6e72eeb6c" Sep 09 23:05:25.624771 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.624751 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 23:05:25.625101 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.625087 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ed2e010-42b2-4939-9a55-f5618b14bb44" containerName="kube-rbac-proxy-web" Sep 09 23:05:25.625156 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.625103 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ed2e010-42b2-4939-9a55-f5618b14bb44" containerName="kube-rbac-proxy-web" Sep 09 23:05:25.625156 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.625117 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c15d7152-f7f5-4d4c-8d66-4f955b5b9725" containerName="registry" Sep 09 23:05:25.625156 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.625122 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c15d7152-f7f5-4d4c-8d66-4f955b5b9725" containerName="registry" Sep 09 23:05:25.625156 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.625130 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ed2e010-42b2-4939-9a55-f5618b14bb44" containerName="prometheus" Sep 09 23:05:25.625156 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.625135 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ed2e010-42b2-4939-9a55-f5618b14bb44" containerName="prometheus" Sep 09 23:05:25.625156 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.625140 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ed2e010-42b2-4939-9a55-f5618b14bb44" containerName="config-reloader" Sep 09 23:05:25.625156 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.625145 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ed2e010-42b2-4939-9a55-f5618b14bb44" containerName="config-reloader" Sep 09 23:05:25.625156 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.625154 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ed2e010-42b2-4939-9a55-f5618b14bb44" containerName="init-config-reloader" Sep 09 23:05:25.625156 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.625159 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ed2e010-42b2-4939-9a55-f5618b14bb44" containerName="init-config-reloader" Sep 09 23:05:25.625417 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.625165 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ed2e010-42b2-4939-9a55-f5618b14bb44" containerName="kube-rbac-proxy" Sep 09 23:05:25.625417 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.625171 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ed2e010-42b2-4939-9a55-f5618b14bb44" containerName="kube-rbac-proxy" Sep 09 23:05:25.625417 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.625177 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ed2e010-42b2-4939-9a55-f5618b14bb44" containerName="kube-rbac-proxy-thanos" Sep 09 23:05:25.625417 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.625182 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ed2e010-42b2-4939-9a55-f5618b14bb44" containerName="kube-rbac-proxy-thanos" Sep 09 23:05:25.625417 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.625189 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ed2e010-42b2-4939-9a55-f5618b14bb44" containerName="thanos-sidecar" Sep 09 23:05:25.625417 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.625194 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ed2e010-42b2-4939-9a55-f5618b14bb44" containerName="thanos-sidecar" Sep 09 23:05:25.625417 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.625242 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ed2e010-42b2-4939-9a55-f5618b14bb44" containerName="prometheus" Sep 09 23:05:25.625417 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.625249 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ed2e010-42b2-4939-9a55-f5618b14bb44" containerName="config-reloader" Sep 09 23:05:25.625417 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.625256 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c15d7152-f7f5-4d4c-8d66-4f955b5b9725" containerName="registry" Sep 09 23:05:25.625417 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.625262 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ed2e010-42b2-4939-9a55-f5618b14bb44" containerName="kube-rbac-proxy-web" Sep 09 23:05:25.625417 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.625267 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ed2e010-42b2-4939-9a55-f5618b14bb44" containerName="kube-rbac-proxy" Sep 09 23:05:25.625417 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.625272 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ed2e010-42b2-4939-9a55-f5618b14bb44" containerName="kube-rbac-proxy-thanos" Sep 09 23:05:25.625417 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.625279 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ed2e010-42b2-4939-9a55-f5618b14bb44" containerName="thanos-sidecar" Sep 09 23:05:25.630416 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.630401 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.633514 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.633287 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 09 23:05:25.633514 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.633357 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 09 23:05:25.633514 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.633375 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 09 23:05:25.633514 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.633362 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 09 23:05:25.633514 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.633394 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 09 23:05:25.633514 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.633408 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 09 23:05:25.633514 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.633382 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 09 23:05:25.633514 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.633378 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-92bjv\"" Sep 09 23:05:25.634128 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.633738 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 09 23:05:25.634128 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.633751 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 09 23:05:25.634128 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.633767 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 09 23:05:25.634128 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.633884 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 09 23:05:25.634328 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.634262 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5jqmv5h73k2hf\"" Sep 09 23:05:25.634616 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.634573 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 09 23:05:25.634845 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.634829 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 09 23:05:25.636213 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.636193 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 09 23:05:25.639023 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.639001 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 09 23:05:25.643627 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.643608 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 23:05:25.742937 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.742870 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2554a324-4a9f-46b0-bb48-099ba2dfe771-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.742937 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.742935 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2554a324-4a9f-46b0-bb48-099ba2dfe771-config-out\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.743159 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.743019 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2554a324-4a9f-46b0-bb48-099ba2dfe771-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.743159 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.743051 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2554a324-4a9f-46b0-bb48-099ba2dfe771-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.743159 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.743068 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2554a324-4a9f-46b0-bb48-099ba2dfe771-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.743159 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.743094 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2554a324-4a9f-46b0-bb48-099ba2dfe771-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.743159 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.743121 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2554a324-4a9f-46b0-bb48-099ba2dfe771-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.743159 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.743154 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2554a324-4a9f-46b0-bb48-099ba2dfe771-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.743363 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.743170 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2554a324-4a9f-46b0-bb48-099ba2dfe771-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.743363 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.743187 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2554a324-4a9f-46b0-bb48-099ba2dfe771-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.743363 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.743203 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2554a324-4a9f-46b0-bb48-099ba2dfe771-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.743363 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.743220 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2554a324-4a9f-46b0-bb48-099ba2dfe771-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.743363 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.743249 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2554a324-4a9f-46b0-bb48-099ba2dfe771-web-config\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.743363 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.743277 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2554a324-4a9f-46b0-bb48-099ba2dfe771-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.743363 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.743315 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5c6q7\" (UniqueName: \"kubernetes.io/projected/2554a324-4a9f-46b0-bb48-099ba2dfe771-kube-api-access-5c6q7\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.743363 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.743340 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2554a324-4a9f-46b0-bb48-099ba2dfe771-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.743363 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.743354 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2554a324-4a9f-46b0-bb48-099ba2dfe771-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.743644 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.743372 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2554a324-4a9f-46b0-bb48-099ba2dfe771-config\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.743644 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.743405 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2554a324-4a9f-46b0-bb48-099ba2dfe771-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.743644 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.743438 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2554a324-4a9f-46b0-bb48-099ba2dfe771-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.844830 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.844754 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2554a324-4a9f-46b0-bb48-099ba2dfe771-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.844830 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.844791 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2554a324-4a9f-46b0-bb48-099ba2dfe771-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.844830 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.844810 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2554a324-4a9f-46b0-bb48-099ba2dfe771-config\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.844830 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.844825 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2554a324-4a9f-46b0-bb48-099ba2dfe771-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.845103 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.844843 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2554a324-4a9f-46b0-bb48-099ba2dfe771-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.845103 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.844862 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2554a324-4a9f-46b0-bb48-099ba2dfe771-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.845103 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.844887 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2554a324-4a9f-46b0-bb48-099ba2dfe771-config-out\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.845103 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.844913 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2554a324-4a9f-46b0-bb48-099ba2dfe771-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.845103 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.844944 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2554a324-4a9f-46b0-bb48-099ba2dfe771-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.845103 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.844994 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2554a324-4a9f-46b0-bb48-099ba2dfe771-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.845103 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.845023 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2554a324-4a9f-46b0-bb48-099ba2dfe771-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.845103 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.845052 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2554a324-4a9f-46b0-bb48-099ba2dfe771-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.845103 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.845088 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2554a324-4a9f-46b0-bb48-099ba2dfe771-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.845103 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.845103 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2554a324-4a9f-46b0-bb48-099ba2dfe771-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.845546 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.845120 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2554a324-4a9f-46b0-bb48-099ba2dfe771-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.845546 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.845135 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2554a324-4a9f-46b0-bb48-099ba2dfe771-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.845546 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.845153 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2554a324-4a9f-46b0-bb48-099ba2dfe771-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.845546 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.845167 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2554a324-4a9f-46b0-bb48-099ba2dfe771-web-config\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.845546 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.845191 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2554a324-4a9f-46b0-bb48-099ba2dfe771-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.845546 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.845225 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5c6q7\" (UniqueName: \"kubernetes.io/projected/2554a324-4a9f-46b0-bb48-099ba2dfe771-kube-api-access-5c6q7\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.845546 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.845254 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2554a324-4a9f-46b0-bb48-099ba2dfe771-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.845903 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.845710 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2554a324-4a9f-46b0-bb48-099ba2dfe771-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.846252 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.846227 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2554a324-4a9f-46b0-bb48-099ba2dfe771-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.846252 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.846242 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2554a324-4a9f-46b0-bb48-099ba2dfe771-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.848369 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.847813 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2554a324-4a9f-46b0-bb48-099ba2dfe771-config-out\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.848369 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.847962 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/2554a324-4a9f-46b0-bb48-099ba2dfe771-config\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.848369 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.848020 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2554a324-4a9f-46b0-bb48-099ba2dfe771-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.848369 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.848241 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2554a324-4a9f-46b0-bb48-099ba2dfe771-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.848921 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.848897 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2554a324-4a9f-46b0-bb48-099ba2dfe771-web-config\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.849028 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.848951 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2554a324-4a9f-46b0-bb48-099ba2dfe771-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.849613 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.849468 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2554a324-4a9f-46b0-bb48-099ba2dfe771-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.849613 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.849538 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2554a324-4a9f-46b0-bb48-099ba2dfe771-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.849613 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.849561 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2554a324-4a9f-46b0-bb48-099ba2dfe771-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.850242 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.850220 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2554a324-4a9f-46b0-bb48-099ba2dfe771-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.850406 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.850388 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2554a324-4a9f-46b0-bb48-099ba2dfe771-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.850543 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.850520 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2554a324-4a9f-46b0-bb48-099ba2dfe771-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.850749 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.850727 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2554a324-4a9f-46b0-bb48-099ba2dfe771-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.850951 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.850936 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2554a324-4a9f-46b0-bb48-099ba2dfe771-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.851534 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.851509 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2554a324-4a9f-46b0-bb48-099ba2dfe771-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.853049 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.853028 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5c6q7\" (UniqueName: \"kubernetes.io/projected/2554a324-4a9f-46b0-bb48-099ba2dfe771-kube-api-access-5c6q7\") pod \"prometheus-k8s-0\" (UID: \"2554a324-4a9f-46b0-bb48-099ba2dfe771\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:25.941492 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:25.941446 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:05:26.221015 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:26.220988 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 23:05:26.223581 ip-10-0-131-92 kubenswrapper[2580]: W0909 23:05:26.223553 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2554a324_4a9f_46b0_bb48_099ba2dfe771.slice/crio-82b5ef4b378e031d631af5f3f690d190719ebfc92b1e21e1975aa05d6e5d98cb WatchSource:0}: Error finding container 82b5ef4b378e031d631af5f3f690d190719ebfc92b1e21e1975aa05d6e5d98cb: Status 404 returned error can't find the container with id 82b5ef4b378e031d631af5f3f690d190719ebfc92b1e21e1975aa05d6e5d98cb Sep 09 23:05:26.580360 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:26.580282 2580 generic.go:356] "Generic (PLEG): container finished" podID="2554a324-4a9f-46b0-bb48-099ba2dfe771" containerID="cd59c7a043709d4ed4519172601ba9cd0c45d290bb9ef800d0a1367d43a92561" exitCode=0 Sep 09 23:05:26.580360 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:26.580351 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2554a324-4a9f-46b0-bb48-099ba2dfe771","Type":"ContainerDied","Data":"cd59c7a043709d4ed4519172601ba9cd0c45d290bb9ef800d0a1367d43a92561"} Sep 09 23:05:26.580714 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:26.580425 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2554a324-4a9f-46b0-bb48-099ba2dfe771","Type":"ContainerStarted","Data":"82b5ef4b378e031d631af5f3f690d190719ebfc92b1e21e1975aa05d6e5d98cb"} Sep 09 23:05:27.179584 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:27.179549 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ed2e010-42b2-4939-9a55-f5618b14bb44" path="/var/lib/kubelet/pods/6ed2e010-42b2-4939-9a55-f5618b14bb44/volumes" Sep 09 23:05:27.587911 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:27.587825 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2554a324-4a9f-46b0-bb48-099ba2dfe771","Type":"ContainerStarted","Data":"9aff771514c810d696d50d8abef86c8cd2be95ab9805996a7da95290e5138c3f"} Sep 09 23:05:27.587911 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:27.587909 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2554a324-4a9f-46b0-bb48-099ba2dfe771","Type":"ContainerStarted","Data":"71559fcec857f06c00d368617ed8c35f23568eee70195893eca67f98abea95dd"} Sep 09 23:05:27.588338 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:27.587948 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2554a324-4a9f-46b0-bb48-099ba2dfe771","Type":"ContainerStarted","Data":"53a16c8f4938de78fbd56a769ad89e5eee20ff563d19dfd26eaa62261938624f"} Sep 09 23:05:27.588338 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:27.588017 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2554a324-4a9f-46b0-bb48-099ba2dfe771","Type":"ContainerStarted","Data":"c1eef6fda9341637a4f15262112d71267e657fed3bc49e4fcaab6451511b6b21"} Sep 09 23:05:27.588338 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:27.588054 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2554a324-4a9f-46b0-bb48-099ba2dfe771","Type":"ContainerStarted","Data":"cbc1e6c16f392a530d73dc7533ad84d312385acfc3071ab15242ed9f9af89ef6"} Sep 09 23:05:27.588338 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:27.588095 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2554a324-4a9f-46b0-bb48-099ba2dfe771","Type":"ContainerStarted","Data":"bd17c02fa4b975f464675a1fd74ae88cf0e544395861e184c92825fb92a0945f"} Sep 09 23:05:27.620343 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:27.620292 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.59957764 podStartE2EDuration="2.620275564s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.697924ms" isStatefulPod=false podCreationTimestamp="2026-09-09 23:05:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:05:27.617399682 +0000 UTC m=+388.967266719" watchObservedRunningTime="2026-09-09 23:05:27.620275564 +0000 UTC m=+388.970142598" Sep 09 23:05:30.942597 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:05:30.942548 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:06:25.941669 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:06:25.941584 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:06:25.957299 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:06:25.957269 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:06:26.838427 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:06:26.838394 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 23:08:25.285334 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:25.285255 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hn2sz"] Sep 09 23:08:25.288552 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:25.288531 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hn2sz" Sep 09 23:08:25.291268 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:25.291241 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Sep 09 23:08:25.292445 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:25.292426 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Sep 09 23:08:25.292507 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:25.292443 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-db2rp\"" Sep 09 23:08:25.302254 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:25.302225 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hn2sz"] Sep 09 23:08:25.460208 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:25.460169 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmp99\" (UniqueName: \"kubernetes.io/projected/10251ad7-f639-4f74-a01e-472453daadbc-kube-api-access-cmp99\") pod \"openshift-lws-operator-fd8ccff4c-hn2sz\" (UID: \"10251ad7-f639-4f74-a01e-472453daadbc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hn2sz" Sep 09 23:08:25.460376 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:25.460294 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/10251ad7-f639-4f74-a01e-472453daadbc-tmp\") pod \"openshift-lws-operator-fd8ccff4c-hn2sz\" (UID: \"10251ad7-f639-4f74-a01e-472453daadbc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hn2sz" Sep 09 23:08:25.561004 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:25.560890 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/10251ad7-f639-4f74-a01e-472453daadbc-tmp\") pod \"openshift-lws-operator-fd8ccff4c-hn2sz\" (UID: \"10251ad7-f639-4f74-a01e-472453daadbc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hn2sz" Sep 09 23:08:25.561004 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:25.560962 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmp99\" (UniqueName: \"kubernetes.io/projected/10251ad7-f639-4f74-a01e-472453daadbc-kube-api-access-cmp99\") pod \"openshift-lws-operator-fd8ccff4c-hn2sz\" (UID: \"10251ad7-f639-4f74-a01e-472453daadbc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hn2sz" Sep 09 23:08:25.561322 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:25.561299 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/10251ad7-f639-4f74-a01e-472453daadbc-tmp\") pod \"openshift-lws-operator-fd8ccff4c-hn2sz\" (UID: \"10251ad7-f639-4f74-a01e-472453daadbc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hn2sz" Sep 09 23:08:25.569449 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:25.569422 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmp99\" (UniqueName: \"kubernetes.io/projected/10251ad7-f639-4f74-a01e-472453daadbc-kube-api-access-cmp99\") pod \"openshift-lws-operator-fd8ccff4c-hn2sz\" (UID: \"10251ad7-f639-4f74-a01e-472453daadbc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hn2sz" Sep 09 23:08:25.612594 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:25.612559 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hn2sz" Sep 09 23:08:25.882197 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:25.882172 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hn2sz"] Sep 09 23:08:25.884491 ip-10-0-131-92 kubenswrapper[2580]: W0909 23:08:25.884460 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod10251ad7_f639_4f74_a01e_472453daadbc.slice/crio-cd1d5a2ca49864123f097e51bc986602d75373036615434659dc2de377f45672 WatchSource:0}: Error finding container cd1d5a2ca49864123f097e51bc986602d75373036615434659dc2de377f45672: Status 404 returned error can't find the container with id cd1d5a2ca49864123f097e51bc986602d75373036615434659dc2de377f45672 Sep 09 23:08:26.293305 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:26.293210 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hn2sz" event={"ID":"10251ad7-f639-4f74-a01e-472453daadbc","Type":"ContainerStarted","Data":"cd1d5a2ca49864123f097e51bc986602d75373036615434659dc2de377f45672"} Sep 09 23:08:29.308804 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:29.308767 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hn2sz" event={"ID":"10251ad7-f639-4f74-a01e-472453daadbc","Type":"ContainerStarted","Data":"f88a1cdeedfbd618727226fc3ef61ceb2b3c6b12c08529de2d710a747936293d"} Sep 09 23:08:29.328650 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:29.328595 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hn2sz" podStartSLOduration=1.927347267 podStartE2EDuration="4.328577101s" totalImagesPullingTime="2.401229834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:08:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:08:29.326272844 +0000 UTC m=+570.676139879" watchObservedRunningTime="2026-09-09 23:08:29.328577101 +0000 UTC m=+570.678444134" Sep 09 23:08:36.541377 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:36.541339 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5c9946bf76-wwkft"] Sep 09 23:08:36.544752 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:36.544732 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5c9946bf76-wwkft" Sep 09 23:08:36.547892 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:36.547868 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Sep 09 23:08:36.548025 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:36.547874 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Sep 09 23:08:36.548025 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:36.547962 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Sep 09 23:08:36.549310 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:36.549295 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-hxlnl\"" Sep 09 23:08:36.553496 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:36.553469 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5c9946bf76-wwkft"] Sep 09 23:08:36.553814 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:36.553795 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d3677ab8-223a-4bb5-bebf-56e5a37b6f37-manager-config\") pod \"lws-controller-manager-5c9946bf76-wwkft\" (UID: \"d3677ab8-223a-4bb5-bebf-56e5a37b6f37\") " pod="openshift-lws-operator/lws-controller-manager-5c9946bf76-wwkft" Sep 09 23:08:36.553887 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:36.553872 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d3677ab8-223a-4bb5-bebf-56e5a37b6f37-metrics-cert\") pod \"lws-controller-manager-5c9946bf76-wwkft\" (UID: \"d3677ab8-223a-4bb5-bebf-56e5a37b6f37\") " pod="openshift-lws-operator/lws-controller-manager-5c9946bf76-wwkft" Sep 09 23:08:36.553951 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:36.553890 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d3677ab8-223a-4bb5-bebf-56e5a37b6f37-cert\") pod \"lws-controller-manager-5c9946bf76-wwkft\" (UID: \"d3677ab8-223a-4bb5-bebf-56e5a37b6f37\") " pod="openshift-lws-operator/lws-controller-manager-5c9946bf76-wwkft" Sep 09 23:08:36.553951 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:36.553913 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnng9\" (UniqueName: \"kubernetes.io/projected/d3677ab8-223a-4bb5-bebf-56e5a37b6f37-kube-api-access-bnng9\") pod \"lws-controller-manager-5c9946bf76-wwkft\" (UID: \"d3677ab8-223a-4bb5-bebf-56e5a37b6f37\") " pod="openshift-lws-operator/lws-controller-manager-5c9946bf76-wwkft" Sep 09 23:08:36.654693 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:36.654652 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d3677ab8-223a-4bb5-bebf-56e5a37b6f37-manager-config\") pod \"lws-controller-manager-5c9946bf76-wwkft\" (UID: \"d3677ab8-223a-4bb5-bebf-56e5a37b6f37\") " pod="openshift-lws-operator/lws-controller-manager-5c9946bf76-wwkft" Sep 09 23:08:36.654887 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:36.654741 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d3677ab8-223a-4bb5-bebf-56e5a37b6f37-metrics-cert\") pod \"lws-controller-manager-5c9946bf76-wwkft\" (UID: \"d3677ab8-223a-4bb5-bebf-56e5a37b6f37\") " pod="openshift-lws-operator/lws-controller-manager-5c9946bf76-wwkft" Sep 09 23:08:36.654887 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:36.654769 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d3677ab8-223a-4bb5-bebf-56e5a37b6f37-cert\") pod \"lws-controller-manager-5c9946bf76-wwkft\" (UID: \"d3677ab8-223a-4bb5-bebf-56e5a37b6f37\") " pod="openshift-lws-operator/lws-controller-manager-5c9946bf76-wwkft" Sep 09 23:08:36.654887 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:36.654796 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnng9\" (UniqueName: \"kubernetes.io/projected/d3677ab8-223a-4bb5-bebf-56e5a37b6f37-kube-api-access-bnng9\") pod \"lws-controller-manager-5c9946bf76-wwkft\" (UID: \"d3677ab8-223a-4bb5-bebf-56e5a37b6f37\") " pod="openshift-lws-operator/lws-controller-manager-5c9946bf76-wwkft" Sep 09 23:08:36.655305 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:36.655283 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d3677ab8-223a-4bb5-bebf-56e5a37b6f37-manager-config\") pod \"lws-controller-manager-5c9946bf76-wwkft\" (UID: \"d3677ab8-223a-4bb5-bebf-56e5a37b6f37\") " pod="openshift-lws-operator/lws-controller-manager-5c9946bf76-wwkft" Sep 09 23:08:36.657219 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:36.657202 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d3677ab8-223a-4bb5-bebf-56e5a37b6f37-metrics-cert\") pod \"lws-controller-manager-5c9946bf76-wwkft\" (UID: \"d3677ab8-223a-4bb5-bebf-56e5a37b6f37\") " pod="openshift-lws-operator/lws-controller-manager-5c9946bf76-wwkft" Sep 09 23:08:36.657339 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:36.657316 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d3677ab8-223a-4bb5-bebf-56e5a37b6f37-cert\") pod \"lws-controller-manager-5c9946bf76-wwkft\" (UID: \"d3677ab8-223a-4bb5-bebf-56e5a37b6f37\") " pod="openshift-lws-operator/lws-controller-manager-5c9946bf76-wwkft" Sep 09 23:08:36.664377 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:36.664353 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnng9\" (UniqueName: \"kubernetes.io/projected/d3677ab8-223a-4bb5-bebf-56e5a37b6f37-kube-api-access-bnng9\") pod \"lws-controller-manager-5c9946bf76-wwkft\" (UID: \"d3677ab8-223a-4bb5-bebf-56e5a37b6f37\") " pod="openshift-lws-operator/lws-controller-manager-5c9946bf76-wwkft" Sep 09 23:08:36.854543 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:36.854462 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5c9946bf76-wwkft" Sep 09 23:08:37.121571 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:37.121546 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5c9946bf76-wwkft"] Sep 09 23:08:37.123682 ip-10-0-131-92 kubenswrapper[2580]: W0909 23:08:37.123645 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3677ab8_223a_4bb5_bebf_56e5a37b6f37.slice/crio-3e703b280c505d760c969664eb8a996753dda783e9917bb1238a00d5347db214 WatchSource:0}: Error finding container 3e703b280c505d760c969664eb8a996753dda783e9917bb1238a00d5347db214: Status 404 returned error can't find the container with id 3e703b280c505d760c969664eb8a996753dda783e9917bb1238a00d5347db214 Sep 09 23:08:37.342341 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:37.342302 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5c9946bf76-wwkft" event={"ID":"d3677ab8-223a-4bb5-bebf-56e5a37b6f37","Type":"ContainerStarted","Data":"3e703b280c505d760c969664eb8a996753dda783e9917bb1238a00d5347db214"} Sep 09 23:08:39.352078 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:39.352039 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5c9946bf76-wwkft" event={"ID":"d3677ab8-223a-4bb5-bebf-56e5a37b6f37","Type":"ContainerStarted","Data":"e4511c25331d6f18a27c60ed24d45f3a3ed3fe27277c2ae5b6d2398d0455a510"} Sep 09 23:08:39.352539 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:39.352125 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5c9946bf76-wwkft" Sep 09 23:08:39.370807 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:39.370758 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5c9946bf76-wwkft" podStartSLOduration=1.921661066 podStartE2EDuration="3.370744026s" totalImagesPullingTime="1.44908296s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:08:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:08:39.369083343 +0000 UTC m=+580.718950377" watchObservedRunningTime="2026-09-09 23:08:39.370744026 +0000 UTC m=+580.720611059" Sep 09 23:08:48.947106 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:48.947071 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65d8796fc4-fbltq"] Sep 09 23:08:48.951622 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:48.951596 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-65d8796fc4-fbltq" Sep 09 23:08:48.954613 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:48.954586 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 09 23:08:48.954714 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:48.954663 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 09 23:08:48.954768 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:48.954750 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-vkrgg\"" Sep 09 23:08:48.954818 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:48.954805 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 09 23:08:48.954859 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:48.954820 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 09 23:08:48.971233 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:48.971211 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65d8796fc4-fbltq"] Sep 09 23:08:49.058163 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:49.058128 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/594f4ec0-c69c-4b20-98aa-90f2fc9bd6f2-manifests\") pod \"opendatahub-operator-controller-manager-65d8796fc4-fbltq\" (UID: \"594f4ec0-c69c-4b20-98aa-90f2fc9bd6f2\") " pod="opendatahub/opendatahub-operator-controller-manager-65d8796fc4-fbltq" Sep 09 23:08:49.058306 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:49.058171 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ld47m\" (UniqueName: \"kubernetes.io/projected/594f4ec0-c69c-4b20-98aa-90f2fc9bd6f2-kube-api-access-ld47m\") pod \"opendatahub-operator-controller-manager-65d8796fc4-fbltq\" (UID: \"594f4ec0-c69c-4b20-98aa-90f2fc9bd6f2\") " pod="opendatahub/opendatahub-operator-controller-manager-65d8796fc4-fbltq" Sep 09 23:08:49.058306 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:49.058232 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/594f4ec0-c69c-4b20-98aa-90f2fc9bd6f2-webhook-cert\") pod \"opendatahub-operator-controller-manager-65d8796fc4-fbltq\" (UID: \"594f4ec0-c69c-4b20-98aa-90f2fc9bd6f2\") " pod="opendatahub/opendatahub-operator-controller-manager-65d8796fc4-fbltq" Sep 09 23:08:49.058306 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:49.058255 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/594f4ec0-c69c-4b20-98aa-90f2fc9bd6f2-tmp\") pod \"opendatahub-operator-controller-manager-65d8796fc4-fbltq\" (UID: \"594f4ec0-c69c-4b20-98aa-90f2fc9bd6f2\") " pod="opendatahub/opendatahub-operator-controller-manager-65d8796fc4-fbltq" Sep 09 23:08:49.058431 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:49.058302 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/594f4ec0-c69c-4b20-98aa-90f2fc9bd6f2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65d8796fc4-fbltq\" (UID: \"594f4ec0-c69c-4b20-98aa-90f2fc9bd6f2\") " pod="opendatahub/opendatahub-operator-controller-manager-65d8796fc4-fbltq" Sep 09 23:08:49.159022 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:49.158988 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/594f4ec0-c69c-4b20-98aa-90f2fc9bd6f2-webhook-cert\") pod \"opendatahub-operator-controller-manager-65d8796fc4-fbltq\" (UID: \"594f4ec0-c69c-4b20-98aa-90f2fc9bd6f2\") " pod="opendatahub/opendatahub-operator-controller-manager-65d8796fc4-fbltq" Sep 09 23:08:49.159184 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:49.159028 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/594f4ec0-c69c-4b20-98aa-90f2fc9bd6f2-tmp\") pod \"opendatahub-operator-controller-manager-65d8796fc4-fbltq\" (UID: \"594f4ec0-c69c-4b20-98aa-90f2fc9bd6f2\") " pod="opendatahub/opendatahub-operator-controller-manager-65d8796fc4-fbltq" Sep 09 23:08:49.159184 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:49.159048 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/594f4ec0-c69c-4b20-98aa-90f2fc9bd6f2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65d8796fc4-fbltq\" (UID: \"594f4ec0-c69c-4b20-98aa-90f2fc9bd6f2\") " pod="opendatahub/opendatahub-operator-controller-manager-65d8796fc4-fbltq" Sep 09 23:08:49.159184 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:49.159092 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/594f4ec0-c69c-4b20-98aa-90f2fc9bd6f2-manifests\") pod \"opendatahub-operator-controller-manager-65d8796fc4-fbltq\" (UID: \"594f4ec0-c69c-4b20-98aa-90f2fc9bd6f2\") " pod="opendatahub/opendatahub-operator-controller-manager-65d8796fc4-fbltq" Sep 09 23:08:49.159184 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:49.159115 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ld47m\" (UniqueName: \"kubernetes.io/projected/594f4ec0-c69c-4b20-98aa-90f2fc9bd6f2-kube-api-access-ld47m\") pod \"opendatahub-operator-controller-manager-65d8796fc4-fbltq\" (UID: \"594f4ec0-c69c-4b20-98aa-90f2fc9bd6f2\") " pod="opendatahub/opendatahub-operator-controller-manager-65d8796fc4-fbltq" Sep 09 23:08:49.159531 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:49.159508 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/594f4ec0-c69c-4b20-98aa-90f2fc9bd6f2-manifests\") pod \"opendatahub-operator-controller-manager-65d8796fc4-fbltq\" (UID: \"594f4ec0-c69c-4b20-98aa-90f2fc9bd6f2\") " pod="opendatahub/opendatahub-operator-controller-manager-65d8796fc4-fbltq" Sep 09 23:08:49.161419 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:49.161397 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/594f4ec0-c69c-4b20-98aa-90f2fc9bd6f2-tmp\") pod \"opendatahub-operator-controller-manager-65d8796fc4-fbltq\" (UID: \"594f4ec0-c69c-4b20-98aa-90f2fc9bd6f2\") " pod="opendatahub/opendatahub-operator-controller-manager-65d8796fc4-fbltq" Sep 09 23:08:49.161527 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:49.161473 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/594f4ec0-c69c-4b20-98aa-90f2fc9bd6f2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65d8796fc4-fbltq\" (UID: \"594f4ec0-c69c-4b20-98aa-90f2fc9bd6f2\") " pod="opendatahub/opendatahub-operator-controller-manager-65d8796fc4-fbltq" Sep 09 23:08:49.161527 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:49.161519 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/594f4ec0-c69c-4b20-98aa-90f2fc9bd6f2-webhook-cert\") pod \"opendatahub-operator-controller-manager-65d8796fc4-fbltq\" (UID: \"594f4ec0-c69c-4b20-98aa-90f2fc9bd6f2\") " pod="opendatahub/opendatahub-operator-controller-manager-65d8796fc4-fbltq" Sep 09 23:08:49.168448 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:49.168421 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ld47m\" (UniqueName: \"kubernetes.io/projected/594f4ec0-c69c-4b20-98aa-90f2fc9bd6f2-kube-api-access-ld47m\") pod \"opendatahub-operator-controller-manager-65d8796fc4-fbltq\" (UID: \"594f4ec0-c69c-4b20-98aa-90f2fc9bd6f2\") " pod="opendatahub/opendatahub-operator-controller-manager-65d8796fc4-fbltq" Sep 09 23:08:49.262133 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:49.262051 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-65d8796fc4-fbltq" Sep 09 23:08:49.556484 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:49.556458 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65d8796fc4-fbltq"] Sep 09 23:08:49.559400 ip-10-0-131-92 kubenswrapper[2580]: W0909 23:08:49.559372 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod594f4ec0_c69c_4b20_98aa_90f2fc9bd6f2.slice/crio-f24eec5626a69561bab4b70e5915d0ddd4d852535716521c86ed6e4f6b5d3a1e WatchSource:0}: Error finding container f24eec5626a69561bab4b70e5915d0ddd4d852535716521c86ed6e4f6b5d3a1e: Status 404 returned error can't find the container with id f24eec5626a69561bab4b70e5915d0ddd4d852535716521c86ed6e4f6b5d3a1e Sep 09 23:08:50.358942 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:50.358907 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5c9946bf76-wwkft" Sep 09 23:08:50.399875 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:50.399831 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65d8796fc4-fbltq" event={"ID":"594f4ec0-c69c-4b20-98aa-90f2fc9bd6f2","Type":"ContainerStarted","Data":"f24eec5626a69561bab4b70e5915d0ddd4d852535716521c86ed6e4f6b5d3a1e"} Sep 09 23:08:53.415103 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:53.415063 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65d8796fc4-fbltq" event={"ID":"594f4ec0-c69c-4b20-98aa-90f2fc9bd6f2","Type":"ContainerStarted","Data":"be602d3b3da0369f8643803f967b051d878db67ff3194d338d70e55c1b9b56b2"} Sep 09 23:08:54.421548 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:54.421512 2580 generic.go:356] "Generic (PLEG): container finished" podID="594f4ec0-c69c-4b20-98aa-90f2fc9bd6f2" containerID="be602d3b3da0369f8643803f967b051d878db67ff3194d338d70e55c1b9b56b2" exitCode=0 Sep 09 23:08:54.421918 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:54.421591 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65d8796fc4-fbltq" event={"ID":"594f4ec0-c69c-4b20-98aa-90f2fc9bd6f2","Type":"ContainerDied","Data":"be602d3b3da0369f8643803f967b051d878db67ff3194d338d70e55c1b9b56b2"} Sep 09 23:08:55.427752 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:55.427719 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65d8796fc4-fbltq" event={"ID":"594f4ec0-c69c-4b20-98aa-90f2fc9bd6f2","Type":"ContainerStarted","Data":"c1a90d364b8f5ba1efad3f226fc072cff47a4377820c8f4744f4f8219e0303fe"} Sep 09 23:08:55.428130 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:55.427852 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-65d8796fc4-fbltq" Sep 09 23:08:55.450871 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:55.450815 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-65d8796fc4-fbltq" podStartSLOduration=2.91374332 podStartE2EDuration="7.450799634s" totalImagesPullingTime="3.458015822s" totalInitContainerRuntime="1.079040492s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:08:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:08:55.449918074 +0000 UTC m=+596.799785119" watchObservedRunningTime="2026-09-09 23:08:55.450799634 +0000 UTC m=+596.800666667" Sep 09 23:08:59.110517 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:59.110486 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xht2x_3b6dd849-6a7e-4141-a460-e9ee0e7ff07c/console-operator/2.log" Sep 09 23:08:59.111051 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:08:59.110765 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xht2x_3b6dd849-6a7e-4141-a460-e9ee0e7ff07c/console-operator/2.log" Sep 09 23:09:06.434734 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:09:06.434701 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-65d8796fc4-fbltq" Sep 09 23:09:08.531327 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:09:08.531298 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7db58bdd4-57gjk"] Sep 09 23:09:08.534483 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:09:08.534465 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7db58bdd4-57gjk" Sep 09 23:09:08.537324 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:09:08.537300 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Sep 09 23:09:08.537439 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:09:08.537306 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-5pswb\"" Sep 09 23:09:08.537610 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:09:08.537594 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Sep 09 23:09:08.546571 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:09:08.546550 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7db58bdd4-57gjk"] Sep 09 23:09:08.628875 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:09:08.628841 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dba3144e-fd21-40d5-a14e-30533456f538-tmp\") pod \"kube-auth-proxy-7db58bdd4-57gjk\" (UID: \"dba3144e-fd21-40d5-a14e-30533456f538\") " pod="openshift-ingress/kube-auth-proxy-7db58bdd4-57gjk" Sep 09 23:09:08.629085 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:09:08.628897 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dba3144e-fd21-40d5-a14e-30533456f538-tls-certs\") pod \"kube-auth-proxy-7db58bdd4-57gjk\" (UID: \"dba3144e-fd21-40d5-a14e-30533456f538\") " pod="openshift-ingress/kube-auth-proxy-7db58bdd4-57gjk" Sep 09 23:09:08.629085 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:09:08.628951 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bz8r8\" (UniqueName: \"kubernetes.io/projected/dba3144e-fd21-40d5-a14e-30533456f538-kube-api-access-bz8r8\") pod \"kube-auth-proxy-7db58bdd4-57gjk\" (UID: \"dba3144e-fd21-40d5-a14e-30533456f538\") " pod="openshift-ingress/kube-auth-proxy-7db58bdd4-57gjk" Sep 09 23:09:08.730188 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:09:08.730150 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dba3144e-fd21-40d5-a14e-30533456f538-tls-certs\") pod \"kube-auth-proxy-7db58bdd4-57gjk\" (UID: \"dba3144e-fd21-40d5-a14e-30533456f538\") " pod="openshift-ingress/kube-auth-proxy-7db58bdd4-57gjk" Sep 09 23:09:08.730188 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:09:08.730189 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bz8r8\" (UniqueName: \"kubernetes.io/projected/dba3144e-fd21-40d5-a14e-30533456f538-kube-api-access-bz8r8\") pod \"kube-auth-proxy-7db58bdd4-57gjk\" (UID: \"dba3144e-fd21-40d5-a14e-30533456f538\") " pod="openshift-ingress/kube-auth-proxy-7db58bdd4-57gjk" Sep 09 23:09:08.730350 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:09:08.730250 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dba3144e-fd21-40d5-a14e-30533456f538-tmp\") pod \"kube-auth-proxy-7db58bdd4-57gjk\" (UID: \"dba3144e-fd21-40d5-a14e-30533456f538\") " pod="openshift-ingress/kube-auth-proxy-7db58bdd4-57gjk" Sep 09 23:09:08.732401 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:09:08.732372 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dba3144e-fd21-40d5-a14e-30533456f538-tmp\") pod \"kube-auth-proxy-7db58bdd4-57gjk\" (UID: \"dba3144e-fd21-40d5-a14e-30533456f538\") " pod="openshift-ingress/kube-auth-proxy-7db58bdd4-57gjk" Sep 09 23:09:08.732494 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:09:08.732466 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dba3144e-fd21-40d5-a14e-30533456f538-tls-certs\") pod \"kube-auth-proxy-7db58bdd4-57gjk\" (UID: \"dba3144e-fd21-40d5-a14e-30533456f538\") " pod="openshift-ingress/kube-auth-proxy-7db58bdd4-57gjk" Sep 09 23:09:08.739481 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:09:08.739457 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bz8r8\" (UniqueName: \"kubernetes.io/projected/dba3144e-fd21-40d5-a14e-30533456f538-kube-api-access-bz8r8\") pod \"kube-auth-proxy-7db58bdd4-57gjk\" (UID: \"dba3144e-fd21-40d5-a14e-30533456f538\") " pod="openshift-ingress/kube-auth-proxy-7db58bdd4-57gjk" Sep 09 23:09:08.844014 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:09:08.843904 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7db58bdd4-57gjk" Sep 09 23:09:09.115400 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:09:09.115376 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7db58bdd4-57gjk"] Sep 09 23:09:09.117735 ip-10-0-131-92 kubenswrapper[2580]: W0909 23:09:09.117701 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddba3144e_fd21_40d5_a14e_30533456f538.slice/crio-0b5bcf3632f91057e4c6b76620609c68c2ee5e69a1a937efa26c5ba54fb9dace WatchSource:0}: Error finding container 0b5bcf3632f91057e4c6b76620609c68c2ee5e69a1a937efa26c5ba54fb9dace: Status 404 returned error can't find the container with id 0b5bcf3632f91057e4c6b76620609c68c2ee5e69a1a937efa26c5ba54fb9dace Sep 09 23:09:09.119492 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:09:09.119472 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 23:09:09.488456 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:09:09.488415 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7db58bdd4-57gjk" event={"ID":"dba3144e-fd21-40d5-a14e-30533456f538","Type":"ContainerStarted","Data":"0b5bcf3632f91057e4c6b76620609c68c2ee5e69a1a937efa26c5ba54fb9dace"} Sep 09 23:09:12.508507 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:09:12.508471 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7db58bdd4-57gjk" event={"ID":"dba3144e-fd21-40d5-a14e-30533456f538","Type":"ContainerStarted","Data":"505e5b73a5738ec137ce58672d11effe02b37c8e2552e8e6da3f5b03ad04332a"} Sep 09 23:09:12.524830 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:09:12.524788 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7db58bdd4-57gjk" podStartSLOduration=1.828261617 podStartE2EDuration="4.524773183s" totalImagesPullingTime="2.696511566s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:09:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:09:12.524277893 +0000 UTC m=+613.874144927" watchObservedRunningTime="2026-09-09 23:09:12.524773183 +0000 UTC m=+613.874640217" Sep 09 23:10:52.014349 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:10:52.014264 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-vtq5h"] Sep 09 23:10:52.017710 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:10:52.017690 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-vtq5h" Sep 09 23:10:52.020720 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:10:52.020698 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Sep 09 23:10:52.020861 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:10:52.020736 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Sep 09 23:10:52.021861 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:10:52.021843 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Sep 09 23:10:52.021963 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:10:52.021843 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Sep 09 23:10:52.021963 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:10:52.021849 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-kr69j\"" Sep 09 23:10:52.024613 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:10:52.024588 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-vtq5h"] Sep 09 23:10:52.123016 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:10:52.122985 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/877ad8ce-df51-40f5-9cef-4ed9ec0b8440-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-vtq5h\" (UID: \"877ad8ce-df51-40f5-9cef-4ed9ec0b8440\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-vtq5h" Sep 09 23:10:52.123183 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:10:52.123025 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xllw\" (UniqueName: \"kubernetes.io/projected/877ad8ce-df51-40f5-9cef-4ed9ec0b8440-kube-api-access-6xllw\") pod \"kuadrant-console-plugin-fbdd4f445-vtq5h\" (UID: \"877ad8ce-df51-40f5-9cef-4ed9ec0b8440\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-vtq5h" Sep 09 23:10:52.123183 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:10:52.123074 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/877ad8ce-df51-40f5-9cef-4ed9ec0b8440-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-vtq5h\" (UID: \"877ad8ce-df51-40f5-9cef-4ed9ec0b8440\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-vtq5h" Sep 09 23:10:52.223505 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:10:52.223460 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/877ad8ce-df51-40f5-9cef-4ed9ec0b8440-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-vtq5h\" (UID: \"877ad8ce-df51-40f5-9cef-4ed9ec0b8440\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-vtq5h" Sep 09 23:10:52.223505 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:10:52.223507 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xllw\" (UniqueName: \"kubernetes.io/projected/877ad8ce-df51-40f5-9cef-4ed9ec0b8440-kube-api-access-6xllw\") pod \"kuadrant-console-plugin-fbdd4f445-vtq5h\" (UID: \"877ad8ce-df51-40f5-9cef-4ed9ec0b8440\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-vtq5h" Sep 09 23:10:52.223716 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:10:52.223562 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/877ad8ce-df51-40f5-9cef-4ed9ec0b8440-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-vtq5h\" (UID: \"877ad8ce-df51-40f5-9cef-4ed9ec0b8440\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-vtq5h" Sep 09 23:10:52.223716 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:10:52.223604 2580 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Sep 09 23:10:52.223716 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:10:52.223684 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/877ad8ce-df51-40f5-9cef-4ed9ec0b8440-plugin-serving-cert podName:877ad8ce-df51-40f5-9cef-4ed9ec0b8440 nodeName:}" failed. No retries permitted until 2026-09-09 23:10:52.723664966 +0000 UTC m=+714.073531977 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/877ad8ce-df51-40f5-9cef-4ed9ec0b8440-plugin-serving-cert") pod "kuadrant-console-plugin-fbdd4f445-vtq5h" (UID: "877ad8ce-df51-40f5-9cef-4ed9ec0b8440") : secret "plugin-serving-cert" not found Sep 09 23:10:52.224210 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:10:52.224193 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/877ad8ce-df51-40f5-9cef-4ed9ec0b8440-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-vtq5h\" (UID: \"877ad8ce-df51-40f5-9cef-4ed9ec0b8440\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-vtq5h" Sep 09 23:10:52.235063 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:10:52.235032 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xllw\" (UniqueName: \"kubernetes.io/projected/877ad8ce-df51-40f5-9cef-4ed9ec0b8440-kube-api-access-6xllw\") pod \"kuadrant-console-plugin-fbdd4f445-vtq5h\" (UID: \"877ad8ce-df51-40f5-9cef-4ed9ec0b8440\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-vtq5h" Sep 09 23:10:52.728076 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:10:52.728035 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/877ad8ce-df51-40f5-9cef-4ed9ec0b8440-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-vtq5h\" (UID: \"877ad8ce-df51-40f5-9cef-4ed9ec0b8440\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-vtq5h" Sep 09 23:10:52.730541 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:10:52.730514 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/877ad8ce-df51-40f5-9cef-4ed9ec0b8440-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-vtq5h\" (UID: \"877ad8ce-df51-40f5-9cef-4ed9ec0b8440\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-vtq5h" Sep 09 23:10:52.927692 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:10:52.927655 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-vtq5h" Sep 09 23:10:53.204191 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:10:53.204093 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-vtq5h"] Sep 09 23:10:53.206798 ip-10-0-131-92 kubenswrapper[2580]: W0909 23:10:53.206772 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod877ad8ce_df51_40f5_9cef_4ed9ec0b8440.slice/crio-3e20fbd63798b92fbd21122cc8196991f97b8439501d44464a7060f65b7830b8 WatchSource:0}: Error finding container 3e20fbd63798b92fbd21122cc8196991f97b8439501d44464a7060f65b7830b8: Status 404 returned error can't find the container with id 3e20fbd63798b92fbd21122cc8196991f97b8439501d44464a7060f65b7830b8 Sep 09 23:10:53.980760 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:10:53.980727 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-vtq5h" event={"ID":"877ad8ce-df51-40f5-9cef-4ed9ec0b8440","Type":"ContainerStarted","Data":"3e20fbd63798b92fbd21122cc8196991f97b8439501d44464a7060f65b7830b8"} Sep 09 23:10:58.003010 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:10:58.002966 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-vtq5h" event={"ID":"877ad8ce-df51-40f5-9cef-4ed9ec0b8440","Type":"ContainerStarted","Data":"0981fc7028f75ddbd1416ef51bfda7108f2c549d4bf47114ace5515976c39c97"} Sep 09 23:10:58.020897 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:10:58.020849 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-vtq5h" podStartSLOduration=2.393168172 podStartE2EDuration="7.020834605s" totalImagesPullingTime="4.627666433s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:10:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:10:58.019671447 +0000 UTC m=+719.369538481" watchObservedRunningTime="2026-09-09 23:10:58.020834605 +0000 UTC m=+719.370701638" Sep 09 23:11:37.165445 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:37.165408 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lfr68"] Sep 09 23:11:37.189770 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:37.189735 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-lfr68" Sep 09 23:11:37.192495 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:37.192464 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Sep 09 23:11:37.197402 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:37.197381 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lfr68"] Sep 09 23:11:37.197474 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:37.197428 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lfr68"] Sep 09 23:11:37.305650 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:37.305603 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/89e5537c-524f-4b19-984d-2132c486ba00-config-file\") pod \"limitador-limitador-84994f7ffc-lfr68\" (UID: \"89e5537c-524f-4b19-984d-2132c486ba00\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-lfr68" Sep 09 23:11:37.305820 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:37.305685 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bhqm\" (UniqueName: \"kubernetes.io/projected/89e5537c-524f-4b19-984d-2132c486ba00-kube-api-access-5bhqm\") pod \"limitador-limitador-84994f7ffc-lfr68\" (UID: \"89e5537c-524f-4b19-984d-2132c486ba00\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-lfr68" Sep 09 23:11:37.406173 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:37.406134 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bhqm\" (UniqueName: \"kubernetes.io/projected/89e5537c-524f-4b19-984d-2132c486ba00-kube-api-access-5bhqm\") pod \"limitador-limitador-84994f7ffc-lfr68\" (UID: \"89e5537c-524f-4b19-984d-2132c486ba00\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-lfr68" Sep 09 23:11:37.406341 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:37.406189 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/89e5537c-524f-4b19-984d-2132c486ba00-config-file\") pod \"limitador-limitador-84994f7ffc-lfr68\" (UID: \"89e5537c-524f-4b19-984d-2132c486ba00\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-lfr68" Sep 09 23:11:37.406744 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:37.406725 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/89e5537c-524f-4b19-984d-2132c486ba00-config-file\") pod \"limitador-limitador-84994f7ffc-lfr68\" (UID: \"89e5537c-524f-4b19-984d-2132c486ba00\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-lfr68" Sep 09 23:11:37.414492 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:37.414463 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bhqm\" (UniqueName: \"kubernetes.io/projected/89e5537c-524f-4b19-984d-2132c486ba00-kube-api-access-5bhqm\") pod \"limitador-limitador-84994f7ffc-lfr68\" (UID: \"89e5537c-524f-4b19-984d-2132c486ba00\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-lfr68" Sep 09 23:11:37.501179 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:37.501086 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-lfr68" Sep 09 23:11:37.615763 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:37.615726 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-qttb7"] Sep 09 23:11:37.619558 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:37.619532 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-qttb7" Sep 09 23:11:37.622113 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:37.622089 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-ll5q8\"" Sep 09 23:11:37.626503 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:37.626479 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-qttb7"] Sep 09 23:11:37.710077 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:37.710034 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qrkb\" (UniqueName: \"kubernetes.io/projected/9057d2ee-5fca-4dd9-9760-628f72cdcf8a-kube-api-access-7qrkb\") pod \"authorino-7bcfd8f9cd-qttb7\" (UID: \"9057d2ee-5fca-4dd9-9760-628f72cdcf8a\") " pod="kuadrant-system/authorino-7bcfd8f9cd-qttb7" Sep 09 23:11:37.775494 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:37.775468 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lfr68"] Sep 09 23:11:37.777478 ip-10-0-131-92 kubenswrapper[2580]: W0909 23:11:37.777454 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89e5537c_524f_4b19_984d_2132c486ba00.slice/crio-12a42be21879b2f1fedac3591a9b111dad15e6996eb9fe0006d0efc2af8fa15a WatchSource:0}: Error finding container 12a42be21879b2f1fedac3591a9b111dad15e6996eb9fe0006d0efc2af8fa15a: Status 404 returned error can't find the container with id 12a42be21879b2f1fedac3591a9b111dad15e6996eb9fe0006d0efc2af8fa15a Sep 09 23:11:37.810699 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:37.810662 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qrkb\" (UniqueName: \"kubernetes.io/projected/9057d2ee-5fca-4dd9-9760-628f72cdcf8a-kube-api-access-7qrkb\") pod \"authorino-7bcfd8f9cd-qttb7\" (UID: \"9057d2ee-5fca-4dd9-9760-628f72cdcf8a\") " pod="kuadrant-system/authorino-7bcfd8f9cd-qttb7" Sep 09 23:11:37.820352 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:37.820318 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qrkb\" (UniqueName: \"kubernetes.io/projected/9057d2ee-5fca-4dd9-9760-628f72cdcf8a-kube-api-access-7qrkb\") pod \"authorino-7bcfd8f9cd-qttb7\" (UID: \"9057d2ee-5fca-4dd9-9760-628f72cdcf8a\") " pod="kuadrant-system/authorino-7bcfd8f9cd-qttb7" Sep 09 23:11:37.930174 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:37.930130 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-qttb7" Sep 09 23:11:38.190055 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:38.190023 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-lfr68" event={"ID":"89e5537c-524f-4b19-984d-2132c486ba00","Type":"ContainerStarted","Data":"12a42be21879b2f1fedac3591a9b111dad15e6996eb9fe0006d0efc2af8fa15a"} Sep 09 23:11:38.198453 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:38.198328 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-qttb7"] Sep 09 23:11:38.200617 ip-10-0-131-92 kubenswrapper[2580]: W0909 23:11:38.200592 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9057d2ee_5fca_4dd9_9760_628f72cdcf8a.slice/crio-974fab1d08485cb0ca29050f881e48100a528e760b63f8955ca234b796bd053b WatchSource:0}: Error finding container 974fab1d08485cb0ca29050f881e48100a528e760b63f8955ca234b796bd053b: Status 404 returned error can't find the container with id 974fab1d08485cb0ca29050f881e48100a528e760b63f8955ca234b796bd053b Sep 09 23:11:39.201081 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:39.201012 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-qttb7" event={"ID":"9057d2ee-5fca-4dd9-9760-628f72cdcf8a","Type":"ContainerStarted","Data":"974fab1d08485cb0ca29050f881e48100a528e760b63f8955ca234b796bd053b"} Sep 09 23:11:40.847882 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:40.847841 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-qttb7"] Sep 09 23:11:42.221578 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:42.221542 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-lfr68" event={"ID":"89e5537c-524f-4b19-984d-2132c486ba00","Type":"ContainerStarted","Data":"3a49d5509acd3fd7d6b44a9e9f21d1ea05e537edd48176b3981ef2ad20252b15"} Sep 09 23:11:42.222136 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:42.221652 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-lfr68" Sep 09 23:11:42.223185 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:42.223162 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-qttb7" event={"ID":"9057d2ee-5fca-4dd9-9760-628f72cdcf8a","Type":"ContainerStarted","Data":"2c8579eb7d4c5fa258a66a7206fc3d71ad8b1883016c2b71f7329ec25d0e2d32"} Sep 09 23:11:42.223280 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:42.223180 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-qttb7" podUID="9057d2ee-5fca-4dd9-9760-628f72cdcf8a" containerName="authorino" containerID="cri-o://2c8579eb7d4c5fa258a66a7206fc3d71ad8b1883016c2b71f7329ec25d0e2d32" gracePeriod=30 Sep 09 23:11:42.239422 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:42.239373 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-lfr68" podStartSLOduration=1.8559128249999999 podStartE2EDuration="5.23935717s" totalImagesPullingTime="3.383444345s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:11:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:11:42.238283637 +0000 UTC m=+763.588150683" watchObservedRunningTime="2026-09-09 23:11:42.23935717 +0000 UTC m=+763.589224204" Sep 09 23:11:42.254038 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:42.253967 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-qttb7" podStartSLOduration=2.287118241 podStartE2EDuration="5.25395079s" totalImagesPullingTime="2.966832549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:11:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:11:42.252668169 +0000 UTC m=+763.602535204" watchObservedRunningTime="2026-09-09 23:11:42.25395079 +0000 UTC m=+763.603817823" Sep 09 23:11:42.466034 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:42.466001 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-qttb7" Sep 09 23:11:42.553685 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:42.553600 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7qrkb\" (UniqueName: \"kubernetes.io/projected/9057d2ee-5fca-4dd9-9760-628f72cdcf8a-kube-api-access-7qrkb\") pod \"9057d2ee-5fca-4dd9-9760-628f72cdcf8a\" (UID: \"9057d2ee-5fca-4dd9-9760-628f72cdcf8a\") " Sep 09 23:11:42.555740 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:42.555708 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9057d2ee-5fca-4dd9-9760-628f72cdcf8a-kube-api-access-7qrkb" (OuterVolumeSpecName: "kube-api-access-7qrkb") pod "9057d2ee-5fca-4dd9-9760-628f72cdcf8a" (UID: "9057d2ee-5fca-4dd9-9760-628f72cdcf8a"). InnerVolumeSpecName "kube-api-access-7qrkb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 23:11:42.654232 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:42.654195 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qrkb\" (UniqueName: \"kubernetes.io/projected/9057d2ee-5fca-4dd9-9760-628f72cdcf8a-kube-api-access-7qrkb\") on node \"ip-10-0-131-92.ec2.internal\" DevicePath \"\"" Sep 09 23:11:43.234918 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:43.234873 2580 generic.go:356] "Generic (PLEG): container finished" podID="9057d2ee-5fca-4dd9-9760-628f72cdcf8a" containerID="2c8579eb7d4c5fa258a66a7206fc3d71ad8b1883016c2b71f7329ec25d0e2d32" exitCode=0 Sep 09 23:11:43.235403 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:43.234986 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-qttb7" Sep 09 23:11:43.235403 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:43.235004 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-qttb7" event={"ID":"9057d2ee-5fca-4dd9-9760-628f72cdcf8a","Type":"ContainerDied","Data":"2c8579eb7d4c5fa258a66a7206fc3d71ad8b1883016c2b71f7329ec25d0e2d32"} Sep 09 23:11:43.235403 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:43.235072 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-qttb7" event={"ID":"9057d2ee-5fca-4dd9-9760-628f72cdcf8a","Type":"ContainerDied","Data":"974fab1d08485cb0ca29050f881e48100a528e760b63f8955ca234b796bd053b"} Sep 09 23:11:43.235403 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:43.235110 2580 scope.go:117] "RemoveContainer" containerID="2c8579eb7d4c5fa258a66a7206fc3d71ad8b1883016c2b71f7329ec25d0e2d32" Sep 09 23:11:43.244219 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:43.244198 2580 scope.go:117] "RemoveContainer" containerID="2c8579eb7d4c5fa258a66a7206fc3d71ad8b1883016c2b71f7329ec25d0e2d32" Sep 09 23:11:43.244529 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:11:43.244512 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c8579eb7d4c5fa258a66a7206fc3d71ad8b1883016c2b71f7329ec25d0e2d32\": container with ID starting with 2c8579eb7d4c5fa258a66a7206fc3d71ad8b1883016c2b71f7329ec25d0e2d32 not found: ID does not exist" containerID="2c8579eb7d4c5fa258a66a7206fc3d71ad8b1883016c2b71f7329ec25d0e2d32" Sep 09 23:11:43.244588 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:43.244540 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c8579eb7d4c5fa258a66a7206fc3d71ad8b1883016c2b71f7329ec25d0e2d32"} err="failed to get container status \"2c8579eb7d4c5fa258a66a7206fc3d71ad8b1883016c2b71f7329ec25d0e2d32\": rpc error: code = NotFound desc = could not find container \"2c8579eb7d4c5fa258a66a7206fc3d71ad8b1883016c2b71f7329ec25d0e2d32\": container with ID starting with 2c8579eb7d4c5fa258a66a7206fc3d71ad8b1883016c2b71f7329ec25d0e2d32 not found: ID does not exist" Sep 09 23:11:43.251470 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:43.251439 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-qttb7"] Sep 09 23:11:43.254863 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:43.254835 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-qttb7"] Sep 09 23:11:45.178854 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:45.178816 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9057d2ee-5fca-4dd9-9760-628f72cdcf8a" path="/var/lib/kubelet/pods/9057d2ee-5fca-4dd9-9760-628f72cdcf8a/volumes" Sep 09 23:11:53.236009 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:11:53.235953 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-lfr68" Sep 09 23:12:11.517917 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:12:11.517881 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-cs6zd"] Sep 09 23:12:11.518398 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:12:11.518254 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9057d2ee-5fca-4dd9-9760-628f72cdcf8a" containerName="authorino" Sep 09 23:12:11.518398 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:12:11.518266 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9057d2ee-5fca-4dd9-9760-628f72cdcf8a" containerName="authorino" Sep 09 23:12:11.518398 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:12:11.518324 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9057d2ee-5fca-4dd9-9760-628f72cdcf8a" containerName="authorino" Sep 09 23:12:11.522731 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:12:11.522710 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-cs6zd" Sep 09 23:12:11.525230 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:12:11.525207 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-ll5q8\"" Sep 09 23:12:11.529386 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:12:11.529364 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-cs6zd"] Sep 09 23:12:11.598433 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:12:11.598402 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7xr8\" (UniqueName: \"kubernetes.io/projected/83b11faa-24b2-4c19-92ef-c912c78bb9e1-kube-api-access-l7xr8\") pod \"authorino-557f876f69-cs6zd\" (UID: \"83b11faa-24b2-4c19-92ef-c912c78bb9e1\") " pod="kuadrant-system/authorino-557f876f69-cs6zd" Sep 09 23:12:11.698844 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:12:11.698809 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7xr8\" (UniqueName: \"kubernetes.io/projected/83b11faa-24b2-4c19-92ef-c912c78bb9e1-kube-api-access-l7xr8\") pod \"authorino-557f876f69-cs6zd\" (UID: \"83b11faa-24b2-4c19-92ef-c912c78bb9e1\") " pod="kuadrant-system/authorino-557f876f69-cs6zd" Sep 09 23:12:11.709505 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:12:11.709478 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7xr8\" (UniqueName: \"kubernetes.io/projected/83b11faa-24b2-4c19-92ef-c912c78bb9e1-kube-api-access-l7xr8\") pod \"authorino-557f876f69-cs6zd\" (UID: \"83b11faa-24b2-4c19-92ef-c912c78bb9e1\") " pod="kuadrant-system/authorino-557f876f69-cs6zd" Sep 09 23:12:11.746961 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:12:11.746930 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-cs6zd"] Sep 09 23:12:11.747192 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:12:11.747179 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-cs6zd" Sep 09 23:12:12.014072 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:12:12.014048 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-cs6zd"] Sep 09 23:12:12.015750 ip-10-0-131-92 kubenswrapper[2580]: W0909 23:12:12.015724 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod83b11faa_24b2_4c19_92ef_c912c78bb9e1.slice/crio-52ab359b31a87ae6db3a92fe575c6ea6cad160847b139184268a789a7dbe92a8 WatchSource:0}: Error finding container 52ab359b31a87ae6db3a92fe575c6ea6cad160847b139184268a789a7dbe92a8: Status 404 returned error can't find the container with id 52ab359b31a87ae6db3a92fe575c6ea6cad160847b139184268a789a7dbe92a8 Sep 09 23:12:12.368860 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:12:12.368822 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-cs6zd" event={"ID":"83b11faa-24b2-4c19-92ef-c912c78bb9e1","Type":"ContainerStarted","Data":"52ab359b31a87ae6db3a92fe575c6ea6cad160847b139184268a789a7dbe92a8"} Sep 09 23:12:13.376129 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:12:13.376096 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-cs6zd" event={"ID":"83b11faa-24b2-4c19-92ef-c912c78bb9e1","Type":"ContainerStarted","Data":"55263f1e519acbdaf044a83ad72df2750b5537618f9f61fa39757c3444389007"} Sep 09 23:12:13.376129 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:12:13.376107 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-557f876f69-cs6zd" podUID="83b11faa-24b2-4c19-92ef-c912c78bb9e1" containerName="authorino" containerID="cri-o://55263f1e519acbdaf044a83ad72df2750b5537618f9f61fa39757c3444389007" gracePeriod=30 Sep 09 23:12:13.392931 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:12:13.392886 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-557f876f69-cs6zd" podStartSLOduration=1.962810017 podStartE2EDuration="2.392870567s" totalImagesPullingTime="430.06055ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:12:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:12:13.391819843 +0000 UTC m=+794.741686877" watchObservedRunningTime="2026-09-09 23:12:13.392870567 +0000 UTC m=+794.742737598" Sep 09 23:12:13.622450 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:12:13.622424 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-cs6zd" Sep 09 23:12:13.716578 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:12:13.716504 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l7xr8\" (UniqueName: \"kubernetes.io/projected/83b11faa-24b2-4c19-92ef-c912c78bb9e1-kube-api-access-l7xr8\") pod \"83b11faa-24b2-4c19-92ef-c912c78bb9e1\" (UID: \"83b11faa-24b2-4c19-92ef-c912c78bb9e1\") " Sep 09 23:12:13.718638 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:12:13.718603 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83b11faa-24b2-4c19-92ef-c912c78bb9e1-kube-api-access-l7xr8" (OuterVolumeSpecName: "kube-api-access-l7xr8") pod "83b11faa-24b2-4c19-92ef-c912c78bb9e1" (UID: "83b11faa-24b2-4c19-92ef-c912c78bb9e1"). InnerVolumeSpecName "kube-api-access-l7xr8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 23:12:13.817936 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:12:13.817902 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7xr8\" (UniqueName: \"kubernetes.io/projected/83b11faa-24b2-4c19-92ef-c912c78bb9e1-kube-api-access-l7xr8\") on node \"ip-10-0-131-92.ec2.internal\" DevicePath \"\"" Sep 09 23:12:14.382139 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:12:14.382101 2580 generic.go:356] "Generic (PLEG): container finished" podID="83b11faa-24b2-4c19-92ef-c912c78bb9e1" containerID="55263f1e519acbdaf044a83ad72df2750b5537618f9f61fa39757c3444389007" exitCode=0 Sep 09 23:12:14.382525 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:12:14.382160 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-cs6zd" Sep 09 23:12:14.382525 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:12:14.382188 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-cs6zd" event={"ID":"83b11faa-24b2-4c19-92ef-c912c78bb9e1","Type":"ContainerDied","Data":"55263f1e519acbdaf044a83ad72df2750b5537618f9f61fa39757c3444389007"} Sep 09 23:12:14.382525 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:12:14.382235 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-cs6zd" event={"ID":"83b11faa-24b2-4c19-92ef-c912c78bb9e1","Type":"ContainerDied","Data":"52ab359b31a87ae6db3a92fe575c6ea6cad160847b139184268a789a7dbe92a8"} Sep 09 23:12:14.382525 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:12:14.382253 2580 scope.go:117] "RemoveContainer" containerID="55263f1e519acbdaf044a83ad72df2750b5537618f9f61fa39757c3444389007" Sep 09 23:12:14.391674 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:12:14.391658 2580 scope.go:117] "RemoveContainer" containerID="55263f1e519acbdaf044a83ad72df2750b5537618f9f61fa39757c3444389007" Sep 09 23:12:14.391940 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:12:14.391919 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55263f1e519acbdaf044a83ad72df2750b5537618f9f61fa39757c3444389007\": container with ID starting with 55263f1e519acbdaf044a83ad72df2750b5537618f9f61fa39757c3444389007 not found: ID does not exist" containerID="55263f1e519acbdaf044a83ad72df2750b5537618f9f61fa39757c3444389007" Sep 09 23:12:14.392045 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:12:14.391946 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55263f1e519acbdaf044a83ad72df2750b5537618f9f61fa39757c3444389007"} err="failed to get container status \"55263f1e519acbdaf044a83ad72df2750b5537618f9f61fa39757c3444389007\": rpc error: code = NotFound desc = could not find container \"55263f1e519acbdaf044a83ad72df2750b5537618f9f61fa39757c3444389007\": container with ID starting with 55263f1e519acbdaf044a83ad72df2750b5537618f9f61fa39757c3444389007 not found: ID does not exist" Sep 09 23:12:14.403057 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:12:14.403026 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-cs6zd"] Sep 09 23:12:14.411322 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:12:14.411300 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-cs6zd"] Sep 09 23:12:15.178810 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:12:15.178779 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83b11faa-24b2-4c19-92ef-c912c78bb9e1" path="/var/lib/kubelet/pods/83b11faa-24b2-4c19-92ef-c912c78bb9e1/volumes" Sep 09 23:12:53.774619 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:12:53.774573 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lfr68"] Sep 09 23:13:24.995377 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:24.995344 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lfr68"] Sep 09 23:13:32.001473 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:32.001434 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr"] Sep 09 23:13:32.001867 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:32.001815 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83b11faa-24b2-4c19-92ef-c912c78bb9e1" containerName="authorino" Sep 09 23:13:32.001867 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:32.001828 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="83b11faa-24b2-4c19-92ef-c912c78bb9e1" containerName="authorino" Sep 09 23:13:32.001937 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:32.001887 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="83b11faa-24b2-4c19-92ef-c912c78bb9e1" containerName="authorino" Sep 09 23:13:32.004463 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:32.004442 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" Sep 09 23:13:32.008303 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:32.008282 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Sep 09 23:13:32.008425 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:32.008285 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 09 23:13:32.008533 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:32.008517 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-cl4wn\"" Sep 09 23:13:32.008605 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:32.008572 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 09 23:13:32.013430 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:32.013401 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr"] Sep 09 23:13:32.106595 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:32.106550 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b622d6dd-9145-4fee-94b1-67a9a4695e80-model-cache\") pod \"e2e-distinct-simulated-kserve-6685d86975-gzhjr\" (UID: \"b622d6dd-9145-4fee-94b1-67a9a4695e80\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" Sep 09 23:13:32.106595 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:32.106594 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqsk6\" (UniqueName: \"kubernetes.io/projected/b622d6dd-9145-4fee-94b1-67a9a4695e80-kube-api-access-pqsk6\") pod \"e2e-distinct-simulated-kserve-6685d86975-gzhjr\" (UID: \"b622d6dd-9145-4fee-94b1-67a9a4695e80\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" Sep 09 23:13:32.106849 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:32.106633 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b622d6dd-9145-4fee-94b1-67a9a4695e80-home\") pod \"e2e-distinct-simulated-kserve-6685d86975-gzhjr\" (UID: \"b622d6dd-9145-4fee-94b1-67a9a4695e80\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" Sep 09 23:13:32.106849 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:32.106745 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b622d6dd-9145-4fee-94b1-67a9a4695e80-dshm\") pod \"e2e-distinct-simulated-kserve-6685d86975-gzhjr\" (UID: \"b622d6dd-9145-4fee-94b1-67a9a4695e80\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" Sep 09 23:13:32.106849 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:32.106804 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b622d6dd-9145-4fee-94b1-67a9a4695e80-tls-certs\") pod \"e2e-distinct-simulated-kserve-6685d86975-gzhjr\" (UID: \"b622d6dd-9145-4fee-94b1-67a9a4695e80\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" Sep 09 23:13:32.106849 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:32.106841 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b622d6dd-9145-4fee-94b1-67a9a4695e80-tmp-dir\") pod \"e2e-distinct-simulated-kserve-6685d86975-gzhjr\" (UID: \"b622d6dd-9145-4fee-94b1-67a9a4695e80\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" Sep 09 23:13:32.207651 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:32.207615 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b622d6dd-9145-4fee-94b1-67a9a4695e80-tls-certs\") pod \"e2e-distinct-simulated-kserve-6685d86975-gzhjr\" (UID: \"b622d6dd-9145-4fee-94b1-67a9a4695e80\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" Sep 09 23:13:32.207651 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:32.207654 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b622d6dd-9145-4fee-94b1-67a9a4695e80-tmp-dir\") pod \"e2e-distinct-simulated-kserve-6685d86975-gzhjr\" (UID: \"b622d6dd-9145-4fee-94b1-67a9a4695e80\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" Sep 09 23:13:32.207914 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:32.207708 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b622d6dd-9145-4fee-94b1-67a9a4695e80-model-cache\") pod \"e2e-distinct-simulated-kserve-6685d86975-gzhjr\" (UID: \"b622d6dd-9145-4fee-94b1-67a9a4695e80\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" Sep 09 23:13:32.207914 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:32.207751 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqsk6\" (UniqueName: \"kubernetes.io/projected/b622d6dd-9145-4fee-94b1-67a9a4695e80-kube-api-access-pqsk6\") pod \"e2e-distinct-simulated-kserve-6685d86975-gzhjr\" (UID: \"b622d6dd-9145-4fee-94b1-67a9a4695e80\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" Sep 09 23:13:32.207914 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:32.207858 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b622d6dd-9145-4fee-94b1-67a9a4695e80-home\") pod \"e2e-distinct-simulated-kserve-6685d86975-gzhjr\" (UID: \"b622d6dd-9145-4fee-94b1-67a9a4695e80\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" Sep 09 23:13:32.208111 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:32.207931 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b622d6dd-9145-4fee-94b1-67a9a4695e80-dshm\") pod \"e2e-distinct-simulated-kserve-6685d86975-gzhjr\" (UID: \"b622d6dd-9145-4fee-94b1-67a9a4695e80\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" Sep 09 23:13:32.208164 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:32.208126 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b622d6dd-9145-4fee-94b1-67a9a4695e80-tmp-dir\") pod \"e2e-distinct-simulated-kserve-6685d86975-gzhjr\" (UID: \"b622d6dd-9145-4fee-94b1-67a9a4695e80\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" Sep 09 23:13:32.208215 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:32.208190 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b622d6dd-9145-4fee-94b1-67a9a4695e80-model-cache\") pod \"e2e-distinct-simulated-kserve-6685d86975-gzhjr\" (UID: \"b622d6dd-9145-4fee-94b1-67a9a4695e80\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" Sep 09 23:13:32.208255 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:32.208217 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b622d6dd-9145-4fee-94b1-67a9a4695e80-home\") pod \"e2e-distinct-simulated-kserve-6685d86975-gzhjr\" (UID: \"b622d6dd-9145-4fee-94b1-67a9a4695e80\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" Sep 09 23:13:32.210012 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:32.209989 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b622d6dd-9145-4fee-94b1-67a9a4695e80-dshm\") pod \"e2e-distinct-simulated-kserve-6685d86975-gzhjr\" (UID: \"b622d6dd-9145-4fee-94b1-67a9a4695e80\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" Sep 09 23:13:32.210293 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:32.210274 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b622d6dd-9145-4fee-94b1-67a9a4695e80-tls-certs\") pod \"e2e-distinct-simulated-kserve-6685d86975-gzhjr\" (UID: \"b622d6dd-9145-4fee-94b1-67a9a4695e80\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" Sep 09 23:13:32.215666 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:32.215646 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqsk6\" (UniqueName: \"kubernetes.io/projected/b622d6dd-9145-4fee-94b1-67a9a4695e80-kube-api-access-pqsk6\") pod \"e2e-distinct-simulated-kserve-6685d86975-gzhjr\" (UID: \"b622d6dd-9145-4fee-94b1-67a9a4695e80\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" Sep 09 23:13:32.315049 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:32.314935 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" Sep 09 23:13:32.587945 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:32.587915 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr"] Sep 09 23:13:32.590226 ip-10-0-131-92 kubenswrapper[2580]: W0909 23:13:32.590196 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb622d6dd_9145_4fee_94b1_67a9a4695e80.slice/crio-dc3c62d90a7fbbacc2c7dd0ffdf963dedf3503e26e5ce1943cc5a1785e772409 WatchSource:0}: Error finding container dc3c62d90a7fbbacc2c7dd0ffdf963dedf3503e26e5ce1943cc5a1785e772409: Status 404 returned error can't find the container with id dc3c62d90a7fbbacc2c7dd0ffdf963dedf3503e26e5ce1943cc5a1785e772409 Sep 09 23:13:32.751257 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:32.751221 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" event={"ID":"b622d6dd-9145-4fee-94b1-67a9a4695e80","Type":"ContainerStarted","Data":"dc3c62d90a7fbbacc2c7dd0ffdf963dedf3503e26e5ce1943cc5a1785e772409"} Sep 09 23:13:32.787665 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:32.787629 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lfr68"] Sep 09 23:13:33.757654 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:33.757620 2580 generic.go:356] "Generic (PLEG): container finished" podID="b622d6dd-9145-4fee-94b1-67a9a4695e80" containerID="b781fabd904f06c256dd30f429f8a853fffa0d8e67b1bfe1caef15d280bfcca9" exitCode=0 Sep 09 23:13:33.758104 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:33.757702 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" event={"ID":"b622d6dd-9145-4fee-94b1-67a9a4695e80","Type":"ContainerDied","Data":"b781fabd904f06c256dd30f429f8a853fffa0d8e67b1bfe1caef15d280bfcca9"} Sep 09 23:13:33.758104 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:33.757920 2580 scope.go:117] "RemoveContainer" containerID="b781fabd904f06c256dd30f429f8a853fffa0d8e67b1bfe1caef15d280bfcca9" Sep 09 23:13:34.764722 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:34.764672 2580 generic.go:356] "Generic (PLEG): container finished" podID="b622d6dd-9145-4fee-94b1-67a9a4695e80" containerID="538706e879db7b388d7feda1bcdfae68d763271ab5d58a14c6f5e3c8b4af57f7" exitCode=0 Sep 09 23:13:34.765159 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:34.764755 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" event={"ID":"b622d6dd-9145-4fee-94b1-67a9a4695e80","Type":"ContainerDied","Data":"538706e879db7b388d7feda1bcdfae68d763271ab5d58a14c6f5e3c8b4af57f7"} Sep 09 23:13:34.765159 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:34.764815 2580 scope.go:117] "RemoveContainer" containerID="b781fabd904f06c256dd30f429f8a853fffa0d8e67b1bfe1caef15d280bfcca9" Sep 09 23:13:34.765159 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:34.765092 2580 scope.go:117] "RemoveContainer" containerID="538706e879db7b388d7feda1bcdfae68d763271ab5d58a14c6f5e3c8b4af57f7" Sep 09 23:13:34.765305 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:13:34.765288 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gzhjr_llm(b622d6dd-9145-4fee-94b1-67a9a4695e80)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" podUID="b622d6dd-9145-4fee-94b1-67a9a4695e80" Sep 09 23:13:35.771029 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:35.771000 2580 scope.go:117] "RemoveContainer" containerID="538706e879db7b388d7feda1bcdfae68d763271ab5d58a14c6f5e3c8b4af57f7" Sep 09 23:13:35.771396 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:13:35.771168 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gzhjr_llm(b622d6dd-9145-4fee-94b1-67a9a4695e80)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" podUID="b622d6dd-9145-4fee-94b1-67a9a4695e80" Sep 09 23:13:42.315359 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:42.315324 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" Sep 09 23:13:42.315734 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:42.315380 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" Sep 09 23:13:42.315734 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:42.315396 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" Sep 09 23:13:42.315734 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:42.315409 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" Sep 09 23:13:42.315834 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:42.315759 2580 scope.go:117] "RemoveContainer" containerID="538706e879db7b388d7feda1bcdfae68d763271ab5d58a14c6f5e3c8b4af57f7" Sep 09 23:13:42.315927 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:13:42.315909 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gzhjr_llm(b622d6dd-9145-4fee-94b1-67a9a4695e80)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" podUID="b622d6dd-9145-4fee-94b1-67a9a4695e80" Sep 09 23:13:43.228642 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:43.228605 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lfr68"] Sep 09 23:13:49.495447 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:49.495411 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lfr68"] Sep 09 23:13:56.173714 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:56.173629 2580 scope.go:117] "RemoveContainer" containerID="538706e879db7b388d7feda1bcdfae68d763271ab5d58a14c6f5e3c8b4af57f7" Sep 09 23:13:56.840685 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:56.840608 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lfr68"] Sep 09 23:13:56.867670 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:56.867635 2580 generic.go:356] "Generic (PLEG): container finished" podID="b622d6dd-9145-4fee-94b1-67a9a4695e80" containerID="700843ef0e2ba9f7266c3391e59b34a5d5022b78289a2bf8614d3d40ea6fa331" exitCode=0 Sep 09 23:13:56.867861 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:56.867708 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" event={"ID":"b622d6dd-9145-4fee-94b1-67a9a4695e80","Type":"ContainerDied","Data":"700843ef0e2ba9f7266c3391e59b34a5d5022b78289a2bf8614d3d40ea6fa331"} Sep 09 23:13:56.867861 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:56.867771 2580 scope.go:117] "RemoveContainer" containerID="538706e879db7b388d7feda1bcdfae68d763271ab5d58a14c6f5e3c8b4af57f7" Sep 09 23:13:56.868160 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:56.868140 2580 scope.go:117] "RemoveContainer" containerID="700843ef0e2ba9f7266c3391e59b34a5d5022b78289a2bf8614d3d40ea6fa331" Sep 09 23:13:56.868363 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:13:56.868342 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gzhjr_llm(b622d6dd-9145-4fee-94b1-67a9a4695e80)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" podUID="b622d6dd-9145-4fee-94b1-67a9a4695e80" Sep 09 23:13:59.142555 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:59.142525 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xht2x_3b6dd849-6a7e-4141-a460-e9ee0e7ff07c/console-operator/2.log" Sep 09 23:13:59.143015 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:13:59.142780 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xht2x_3b6dd849-6a7e-4141-a460-e9ee0e7ff07c/console-operator/2.log" Sep 09 23:14:02.315254 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:02.315210 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" Sep 09 23:14:02.315648 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:02.315284 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" Sep 09 23:14:02.315648 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:02.315300 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" Sep 09 23:14:02.315648 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:02.315335 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" Sep 09 23:14:02.315748 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:02.315681 2580 scope.go:117] "RemoveContainer" containerID="700843ef0e2ba9f7266c3391e59b34a5d5022b78289a2bf8614d3d40ea6fa331" Sep 09 23:14:02.315861 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:14:02.315842 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gzhjr_llm(b622d6dd-9145-4fee-94b1-67a9a4695e80)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" podUID="b622d6dd-9145-4fee-94b1-67a9a4695e80" Sep 09 23:14:05.535645 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:05.535609 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lfr68"] Sep 09 23:14:18.174220 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:18.174182 2580 scope.go:117] "RemoveContainer" containerID="700843ef0e2ba9f7266c3391e59b34a5d5022b78289a2bf8614d3d40ea6fa331" Sep 09 23:14:18.177673 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:18.174480 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 23:14:18.976212 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:18.976174 2580 generic.go:356] "Generic (PLEG): container finished" podID="b622d6dd-9145-4fee-94b1-67a9a4695e80" containerID="6c7396d4eab1664b1d23c6210b0f03338adafb0436e582055dbb54c4f50549bd" exitCode=0 Sep 09 23:14:18.976384 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:18.976249 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" event={"ID":"b622d6dd-9145-4fee-94b1-67a9a4695e80","Type":"ContainerDied","Data":"6c7396d4eab1664b1d23c6210b0f03338adafb0436e582055dbb54c4f50549bd"} Sep 09 23:14:18.976384 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:18.976319 2580 scope.go:117] "RemoveContainer" containerID="700843ef0e2ba9f7266c3391e59b34a5d5022b78289a2bf8614d3d40ea6fa331" Sep 09 23:14:18.976689 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:18.976674 2580 scope.go:117] "RemoveContainer" containerID="6c7396d4eab1664b1d23c6210b0f03338adafb0436e582055dbb54c4f50549bd" Sep 09 23:14:18.976882 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:14:18.976864 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gzhjr_llm(b622d6dd-9145-4fee-94b1-67a9a4695e80)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" podUID="b622d6dd-9145-4fee-94b1-67a9a4695e80" Sep 09 23:14:22.071746 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:22.071705 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x"] Sep 09 23:14:22.076573 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:22.076550 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" Sep 09 23:14:22.079454 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:22.079429 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Sep 09 23:14:22.089438 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:22.089413 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x"] Sep 09 23:14:22.225951 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:22.225905 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce8be9ac-786f-4f35-8b40-be4cd36fcb3e-home\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x\" (UID: \"ce8be9ac-786f-4f35-8b40-be4cd36fcb3e\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" Sep 09 23:14:22.226181 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:22.225967 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce8be9ac-786f-4f35-8b40-be4cd36fcb3e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x\" (UID: \"ce8be9ac-786f-4f35-8b40-be4cd36fcb3e\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" Sep 09 23:14:22.226181 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:22.226048 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce8be9ac-786f-4f35-8b40-be4cd36fcb3e-dshm\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x\" (UID: \"ce8be9ac-786f-4f35-8b40-be4cd36fcb3e\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" Sep 09 23:14:22.226181 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:22.226088 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5rvk\" (UniqueName: \"kubernetes.io/projected/ce8be9ac-786f-4f35-8b40-be4cd36fcb3e-kube-api-access-v5rvk\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x\" (UID: \"ce8be9ac-786f-4f35-8b40-be4cd36fcb3e\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" Sep 09 23:14:22.226181 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:22.226162 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce8be9ac-786f-4f35-8b40-be4cd36fcb3e-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x\" (UID: \"ce8be9ac-786f-4f35-8b40-be4cd36fcb3e\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" Sep 09 23:14:22.226348 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:22.226197 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce8be9ac-786f-4f35-8b40-be4cd36fcb3e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x\" (UID: \"ce8be9ac-786f-4f35-8b40-be4cd36fcb3e\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" Sep 09 23:14:22.315744 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:22.315692 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" Sep 09 23:14:22.315920 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:22.315766 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" Sep 09 23:14:22.315920 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:22.315783 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" Sep 09 23:14:22.315920 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:22.315799 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" Sep 09 23:14:22.316215 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:22.316200 2580 scope.go:117] "RemoveContainer" containerID="6c7396d4eab1664b1d23c6210b0f03338adafb0436e582055dbb54c4f50549bd" Sep 09 23:14:22.316412 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:14:22.316394 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gzhjr_llm(b622d6dd-9145-4fee-94b1-67a9a4695e80)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" podUID="b622d6dd-9145-4fee-94b1-67a9a4695e80" Sep 09 23:14:22.327623 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:22.327551 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce8be9ac-786f-4f35-8b40-be4cd36fcb3e-home\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x\" (UID: \"ce8be9ac-786f-4f35-8b40-be4cd36fcb3e\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" Sep 09 23:14:22.327623 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:22.327605 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce8be9ac-786f-4f35-8b40-be4cd36fcb3e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x\" (UID: \"ce8be9ac-786f-4f35-8b40-be4cd36fcb3e\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" Sep 09 23:14:22.327825 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:22.327639 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce8be9ac-786f-4f35-8b40-be4cd36fcb3e-dshm\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x\" (UID: \"ce8be9ac-786f-4f35-8b40-be4cd36fcb3e\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" Sep 09 23:14:22.327825 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:22.327658 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5rvk\" (UniqueName: \"kubernetes.io/projected/ce8be9ac-786f-4f35-8b40-be4cd36fcb3e-kube-api-access-v5rvk\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x\" (UID: \"ce8be9ac-786f-4f35-8b40-be4cd36fcb3e\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" Sep 09 23:14:22.327825 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:22.327689 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce8be9ac-786f-4f35-8b40-be4cd36fcb3e-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x\" (UID: \"ce8be9ac-786f-4f35-8b40-be4cd36fcb3e\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" Sep 09 23:14:22.327825 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:22.327713 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce8be9ac-786f-4f35-8b40-be4cd36fcb3e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x\" (UID: \"ce8be9ac-786f-4f35-8b40-be4cd36fcb3e\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" Sep 09 23:14:22.328085 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:22.328060 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce8be9ac-786f-4f35-8b40-be4cd36fcb3e-home\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x\" (UID: \"ce8be9ac-786f-4f35-8b40-be4cd36fcb3e\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" Sep 09 23:14:22.328135 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:22.328113 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce8be9ac-786f-4f35-8b40-be4cd36fcb3e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x\" (UID: \"ce8be9ac-786f-4f35-8b40-be4cd36fcb3e\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" Sep 09 23:14:22.328176 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:22.328165 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce8be9ac-786f-4f35-8b40-be4cd36fcb3e-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x\" (UID: \"ce8be9ac-786f-4f35-8b40-be4cd36fcb3e\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" Sep 09 23:14:22.330112 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:22.330087 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce8be9ac-786f-4f35-8b40-be4cd36fcb3e-dshm\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x\" (UID: \"ce8be9ac-786f-4f35-8b40-be4cd36fcb3e\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" Sep 09 23:14:22.330260 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:22.330244 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce8be9ac-786f-4f35-8b40-be4cd36fcb3e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x\" (UID: \"ce8be9ac-786f-4f35-8b40-be4cd36fcb3e\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" Sep 09 23:14:22.339986 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:22.339941 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5rvk\" (UniqueName: \"kubernetes.io/projected/ce8be9ac-786f-4f35-8b40-be4cd36fcb3e-kube-api-access-v5rvk\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x\" (UID: \"ce8be9ac-786f-4f35-8b40-be4cd36fcb3e\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" Sep 09 23:14:22.385587 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:22.385535 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" Sep 09 23:14:22.670505 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:22.670468 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x"] Sep 09 23:14:22.674230 ip-10-0-131-92 kubenswrapper[2580]: W0909 23:14:22.674202 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podce8be9ac_786f_4f35_8b40_be4cd36fcb3e.slice/crio-ead13219b4eadf545a144135071c703978288eeb24c5177655db6adc472e726d WatchSource:0}: Error finding container ead13219b4eadf545a144135071c703978288eeb24c5177655db6adc472e726d: Status 404 returned error can't find the container with id ead13219b4eadf545a144135071c703978288eeb24c5177655db6adc472e726d Sep 09 23:14:22.997335 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:22.997300 2580 generic.go:356] "Generic (PLEG): container finished" podID="ce8be9ac-786f-4f35-8b40-be4cd36fcb3e" containerID="aba1a4d88e26414081e6dcb6d6aed4daeede3e9bfa6a0bc747c2c83e23caa29c" exitCode=0 Sep 09 23:14:22.997501 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:22.997387 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" event={"ID":"ce8be9ac-786f-4f35-8b40-be4cd36fcb3e","Type":"ContainerDied","Data":"aba1a4d88e26414081e6dcb6d6aed4daeede3e9bfa6a0bc747c2c83e23caa29c"} Sep 09 23:14:22.997501 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:22.997437 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" event={"ID":"ce8be9ac-786f-4f35-8b40-be4cd36fcb3e","Type":"ContainerStarted","Data":"ead13219b4eadf545a144135071c703978288eeb24c5177655db6adc472e726d"} Sep 09 23:14:22.997620 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:22.997605 2580 scope.go:117] "RemoveContainer" containerID="aba1a4d88e26414081e6dcb6d6aed4daeede3e9bfa6a0bc747c2c83e23caa29c" Sep 09 23:14:24.003890 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:24.003854 2580 generic.go:356] "Generic (PLEG): container finished" podID="ce8be9ac-786f-4f35-8b40-be4cd36fcb3e" containerID="abcea8a7587cbfc066460741230f86c4b536edaad23e37890c8685fff27e947a" exitCode=0 Sep 09 23:14:24.004380 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:24.003899 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" event={"ID":"ce8be9ac-786f-4f35-8b40-be4cd36fcb3e","Type":"ContainerDied","Data":"abcea8a7587cbfc066460741230f86c4b536edaad23e37890c8685fff27e947a"} Sep 09 23:14:24.004380 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:24.003931 2580 scope.go:117] "RemoveContainer" containerID="aba1a4d88e26414081e6dcb6d6aed4daeede3e9bfa6a0bc747c2c83e23caa29c" Sep 09 23:14:24.004380 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:24.004234 2580 scope.go:117] "RemoveContainer" containerID="abcea8a7587cbfc066460741230f86c4b536edaad23e37890c8685fff27e947a" Sep 09 23:14:24.004522 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:14:24.004453 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x_llm(ce8be9ac-786f-4f35-8b40-be4cd36fcb3e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" podUID="ce8be9ac-786f-4f35-8b40-be4cd36fcb3e" Sep 09 23:14:24.215113 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:24.215081 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lfr68"] Sep 09 23:14:25.011092 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:25.011064 2580 scope.go:117] "RemoveContainer" containerID="abcea8a7587cbfc066460741230f86c4b536edaad23e37890c8685fff27e947a" Sep 09 23:14:25.011459 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:14:25.011227 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x_llm(ce8be9ac-786f-4f35-8b40-be4cd36fcb3e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" podUID="ce8be9ac-786f-4f35-8b40-be4cd36fcb3e" Sep 09 23:14:32.385928 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:32.385885 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" Sep 09 23:14:32.386426 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:32.385946 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" Sep 09 23:14:32.386426 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:32.385959 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" Sep 09 23:14:32.386426 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:32.385969 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" Sep 09 23:14:32.386426 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:32.386373 2580 scope.go:117] "RemoveContainer" containerID="abcea8a7587cbfc066460741230f86c4b536edaad23e37890c8685fff27e947a" Sep 09 23:14:32.386568 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:14:32.386532 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x_llm(ce8be9ac-786f-4f35-8b40-be4cd36fcb3e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" podUID="ce8be9ac-786f-4f35-8b40-be4cd36fcb3e" Sep 09 23:14:36.173731 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:36.173699 2580 scope.go:117] "RemoveContainer" containerID="6c7396d4eab1664b1d23c6210b0f03338adafb0436e582055dbb54c4f50549bd" Sep 09 23:14:36.174117 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:14:36.173902 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gzhjr_llm(b622d6dd-9145-4fee-94b1-67a9a4695e80)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" podUID="b622d6dd-9145-4fee-94b1-67a9a4695e80" Sep 09 23:14:47.173368 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:47.173330 2580 scope.go:117] "RemoveContainer" containerID="abcea8a7587cbfc066460741230f86c4b536edaad23e37890c8685fff27e947a" Sep 09 23:14:48.127545 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:48.127514 2580 generic.go:356] "Generic (PLEG): container finished" podID="ce8be9ac-786f-4f35-8b40-be4cd36fcb3e" containerID="f89ea5ff9819b5b25dd97be54968c9a7351a867320bb26bba3e0a21ae95127c4" exitCode=0 Sep 09 23:14:48.127709 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:48.127588 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" event={"ID":"ce8be9ac-786f-4f35-8b40-be4cd36fcb3e","Type":"ContainerDied","Data":"f89ea5ff9819b5b25dd97be54968c9a7351a867320bb26bba3e0a21ae95127c4"} Sep 09 23:14:48.127709 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:48.127634 2580 scope.go:117] "RemoveContainer" containerID="abcea8a7587cbfc066460741230f86c4b536edaad23e37890c8685fff27e947a" Sep 09 23:14:48.127919 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:48.127903 2580 scope.go:117] "RemoveContainer" containerID="f89ea5ff9819b5b25dd97be54968c9a7351a867320bb26bba3e0a21ae95127c4" Sep 09 23:14:48.128130 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:14:48.128109 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x_llm(ce8be9ac-786f-4f35-8b40-be4cd36fcb3e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" podUID="ce8be9ac-786f-4f35-8b40-be4cd36fcb3e" Sep 09 23:14:50.173825 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:50.173795 2580 scope.go:117] "RemoveContainer" containerID="6c7396d4eab1664b1d23c6210b0f03338adafb0436e582055dbb54c4f50549bd" Sep 09 23:14:50.174223 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:14:50.173993 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gzhjr_llm(b622d6dd-9145-4fee-94b1-67a9a4695e80)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" podUID="b622d6dd-9145-4fee-94b1-67a9a4695e80" Sep 09 23:14:52.385888 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:52.385846 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" Sep 09 23:14:52.385888 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:52.385892 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" Sep 09 23:14:52.386336 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:52.385911 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" Sep 09 23:14:52.386336 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:52.385928 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" Sep 09 23:14:52.386336 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:52.386283 2580 scope.go:117] "RemoveContainer" containerID="f89ea5ff9819b5b25dd97be54968c9a7351a867320bb26bba3e0a21ae95127c4" Sep 09 23:14:52.386484 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:14:52.386464 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x_llm(ce8be9ac-786f-4f35-8b40-be4cd36fcb3e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" podUID="ce8be9ac-786f-4f35-8b40-be4cd36fcb3e" Sep 09 23:14:53.153505 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:14:53.153477 2580 scope.go:117] "RemoveContainer" containerID="f89ea5ff9819b5b25dd97be54968c9a7351a867320bb26bba3e0a21ae95127c4" Sep 09 23:14:53.153674 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:14:53.153636 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x_llm(ce8be9ac-786f-4f35-8b40-be4cd36fcb3e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" podUID="ce8be9ac-786f-4f35-8b40-be4cd36fcb3e" Sep 09 23:15:00.144382 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:15:00.144344 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29816595-k9f2g"] Sep 09 23:15:00.147631 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:15:00.147613 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816595-k9f2g" Sep 09 23:15:00.150403 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:15:00.150372 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9kgn4\"" Sep 09 23:15:00.155746 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:15:00.155721 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29816595-k9f2g"] Sep 09 23:15:00.158331 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:15:00.158312 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816595-k9f2g" Sep 09 23:15:00.427675 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:15:00.427647 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29816595-k9f2g"] Sep 09 23:15:00.430018 ip-10-0-131-92 kubenswrapper[2580]: W0909 23:15:00.429989 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25f247f6_fb49_4744_8d2d_40a1391049cc.slice/crio-d8a6cb9493972e690beff916d68cfcff8d26311c56a776ad0447f80cbc355781 WatchSource:0}: Error finding container d8a6cb9493972e690beff916d68cfcff8d26311c56a776ad0447f80cbc355781: Status 404 returned error can't find the container with id d8a6cb9493972e690beff916d68cfcff8d26311c56a776ad0447f80cbc355781 Sep 09 23:15:01.173400 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:15:01.173369 2580 scope.go:117] "RemoveContainer" containerID="6c7396d4eab1664b1d23c6210b0f03338adafb0436e582055dbb54c4f50549bd" Sep 09 23:15:01.191577 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:15:01.191547 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816595-k9f2g" event={"ID":"25f247f6-fb49-4744-8d2d-40a1391049cc","Type":"ContainerStarted","Data":"d8a6cb9493972e690beff916d68cfcff8d26311c56a776ad0447f80cbc355781"} Sep 09 23:15:02.199563 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:15:02.199527 2580 generic.go:356] "Generic (PLEG): container finished" podID="b622d6dd-9145-4fee-94b1-67a9a4695e80" containerID="c9eac83e1a48b696f860a068f7accca5a4978020e5018effea82ac1d001901c9" exitCode=0 Sep 09 23:15:02.200037 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:15:02.199578 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" event={"ID":"b622d6dd-9145-4fee-94b1-67a9a4695e80","Type":"ContainerDied","Data":"c9eac83e1a48b696f860a068f7accca5a4978020e5018effea82ac1d001901c9"} Sep 09 23:15:02.200037 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:15:02.199633 2580 scope.go:117] "RemoveContainer" containerID="6c7396d4eab1664b1d23c6210b0f03338adafb0436e582055dbb54c4f50549bd" Sep 09 23:15:02.200037 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:15:02.200030 2580 scope.go:117] "RemoveContainer" containerID="c9eac83e1a48b696f860a068f7accca5a4978020e5018effea82ac1d001901c9" Sep 09 23:15:02.200279 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:15:02.200255 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gzhjr_llm(b622d6dd-9145-4fee-94b1-67a9a4695e80)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" podUID="b622d6dd-9145-4fee-94b1-67a9a4695e80" Sep 09 23:15:02.316022 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:15:02.315954 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" Sep 09 23:15:02.316189 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:15:02.316061 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" Sep 09 23:15:02.316189 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:15:02.316084 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" Sep 09 23:15:02.316189 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:15:02.316104 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" Sep 09 23:15:03.206547 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:15:03.206525 2580 scope.go:117] "RemoveContainer" containerID="c9eac83e1a48b696f860a068f7accca5a4978020e5018effea82ac1d001901c9" Sep 09 23:15:03.206861 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:15:03.206706 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gzhjr_llm(b622d6dd-9145-4fee-94b1-67a9a4695e80)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" podUID="b622d6dd-9145-4fee-94b1-67a9a4695e80" Sep 09 23:15:04.212521 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:15:04.212485 2580 generic.go:356] "Generic (PLEG): container finished" podID="25f247f6-fb49-4744-8d2d-40a1391049cc" containerID="72dd057d95d7f68149ababa6626b4793046577564ba3305fe8f160bc924d2177" exitCode=0 Sep 09 23:15:04.212907 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:15:04.212571 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816595-k9f2g" event={"ID":"25f247f6-fb49-4744-8d2d-40a1391049cc","Type":"ContainerDied","Data":"72dd057d95d7f68149ababa6626b4793046577564ba3305fe8f160bc924d2177"} Sep 09 23:15:05.173591 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:15:05.173562 2580 scope.go:117] "RemoveContainer" containerID="f89ea5ff9819b5b25dd97be54968c9a7351a867320bb26bba3e0a21ae95127c4" Sep 09 23:15:05.173759 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:15:05.173721 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x_llm(ce8be9ac-786f-4f35-8b40-be4cd36fcb3e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" podUID="ce8be9ac-786f-4f35-8b40-be4cd36fcb3e" Sep 09 23:15:05.343392 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:15:05.343353 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816595-k9f2g" Sep 09 23:15:06.222839 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:15:06.222789 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816595-k9f2g" Sep 09 23:15:06.222839 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:15:06.222826 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816595-k9f2g" event={"ID":"25f247f6-fb49-4744-8d2d-40a1391049cc","Type":"ContainerDied","Data":"d8a6cb9493972e690beff916d68cfcff8d26311c56a776ad0447f80cbc355781"} Sep 09 23:15:06.223060 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:15:06.222875 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d8a6cb9493972e690beff916d68cfcff8d26311c56a776ad0447f80cbc355781" Sep 09 23:15:16.174268 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:15:16.174232 2580 scope.go:117] "RemoveContainer" containerID="c9eac83e1a48b696f860a068f7accca5a4978020e5018effea82ac1d001901c9" Sep 09 23:15:16.174717 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:15:16.174392 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gzhjr_llm(b622d6dd-9145-4fee-94b1-67a9a4695e80)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" podUID="b622d6dd-9145-4fee-94b1-67a9a4695e80" Sep 09 23:15:18.173607 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:15:18.173572 2580 scope.go:117] "RemoveContainer" containerID="f89ea5ff9819b5b25dd97be54968c9a7351a867320bb26bba3e0a21ae95127c4" Sep 09 23:15:19.284618 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:15:19.284581 2580 generic.go:356] "Generic (PLEG): container finished" podID="ce8be9ac-786f-4f35-8b40-be4cd36fcb3e" containerID="eefbe615e947593e2cd611156f2f0451a6cb841e5baf78743b0bb66d4f307c8d" exitCode=0 Sep 09 23:15:19.285082 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:15:19.284634 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" event={"ID":"ce8be9ac-786f-4f35-8b40-be4cd36fcb3e","Type":"ContainerDied","Data":"eefbe615e947593e2cd611156f2f0451a6cb841e5baf78743b0bb66d4f307c8d"} Sep 09 23:15:19.285082 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:15:19.284673 2580 scope.go:117] "RemoveContainer" containerID="f89ea5ff9819b5b25dd97be54968c9a7351a867320bb26bba3e0a21ae95127c4" Sep 09 23:15:19.285082 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:15:19.285062 2580 scope.go:117] "RemoveContainer" containerID="eefbe615e947593e2cd611156f2f0451a6cb841e5baf78743b0bb66d4f307c8d" Sep 09 23:15:19.285282 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:15:19.285262 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x_llm(ce8be9ac-786f-4f35-8b40-be4cd36fcb3e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" podUID="ce8be9ac-786f-4f35-8b40-be4cd36fcb3e" Sep 09 23:15:22.386118 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:15:22.386007 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" Sep 09 23:15:22.386118 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:15:22.386067 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" Sep 09 23:15:22.386118 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:15:22.386097 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" Sep 09 23:15:22.386118 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:15:22.386107 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" Sep 09 23:15:22.386576 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:15:22.386405 2580 scope.go:117] "RemoveContainer" containerID="eefbe615e947593e2cd611156f2f0451a6cb841e5baf78743b0bb66d4f307c8d" Sep 09 23:15:22.386576 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:15:22.386555 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x_llm(ce8be9ac-786f-4f35-8b40-be4cd36fcb3e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" podUID="ce8be9ac-786f-4f35-8b40-be4cd36fcb3e" Sep 09 23:15:30.173911 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:15:30.173880 2580 scope.go:117] "RemoveContainer" containerID="c9eac83e1a48b696f860a068f7accca5a4978020e5018effea82ac1d001901c9" Sep 09 23:15:30.174334 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:15:30.174053 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gzhjr_llm(b622d6dd-9145-4fee-94b1-67a9a4695e80)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" podUID="b622d6dd-9145-4fee-94b1-67a9a4695e80" Sep 09 23:15:34.173536 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:15:34.173508 2580 scope.go:117] "RemoveContainer" containerID="eefbe615e947593e2cd611156f2f0451a6cb841e5baf78743b0bb66d4f307c8d" Sep 09 23:15:34.173887 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:15:34.173652 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x_llm(ce8be9ac-786f-4f35-8b40-be4cd36fcb3e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" podUID="ce8be9ac-786f-4f35-8b40-be4cd36fcb3e" Sep 09 23:15:45.174222 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:15:45.174187 2580 scope.go:117] "RemoveContainer" containerID="eefbe615e947593e2cd611156f2f0451a6cb841e5baf78743b0bb66d4f307c8d" Sep 09 23:15:45.174684 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:15:45.174322 2580 scope.go:117] "RemoveContainer" containerID="c9eac83e1a48b696f860a068f7accca5a4978020e5018effea82ac1d001901c9" Sep 09 23:15:45.174684 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:15:45.174343 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x_llm(ce8be9ac-786f-4f35-8b40-be4cd36fcb3e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" podUID="ce8be9ac-786f-4f35-8b40-be4cd36fcb3e" Sep 09 23:15:45.174684 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:15:45.174515 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gzhjr_llm(b622d6dd-9145-4fee-94b1-67a9a4695e80)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" podUID="b622d6dd-9145-4fee-94b1-67a9a4695e80" Sep 09 23:15:56.173668 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:15:56.173633 2580 scope.go:117] "RemoveContainer" containerID="eefbe615e947593e2cd611156f2f0451a6cb841e5baf78743b0bb66d4f307c8d" Sep 09 23:15:56.174213 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:15:56.173851 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x_llm(ce8be9ac-786f-4f35-8b40-be4cd36fcb3e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" podUID="ce8be9ac-786f-4f35-8b40-be4cd36fcb3e" Sep 09 23:15:58.173789 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:15:58.173749 2580 scope.go:117] "RemoveContainer" containerID="c9eac83e1a48b696f860a068f7accca5a4978020e5018effea82ac1d001901c9" Sep 09 23:15:58.174203 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:15:58.173950 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gzhjr_llm(b622d6dd-9145-4fee-94b1-67a9a4695e80)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" podUID="b622d6dd-9145-4fee-94b1-67a9a4695e80" Sep 09 23:16:07.173532 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:16:07.173501 2580 scope.go:117] "RemoveContainer" containerID="eefbe615e947593e2cd611156f2f0451a6cb841e5baf78743b0bb66d4f307c8d" Sep 09 23:16:07.510648 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:16:07.510615 2580 generic.go:356] "Generic (PLEG): container finished" podID="ce8be9ac-786f-4f35-8b40-be4cd36fcb3e" containerID="9d885f4a27544bafa5fd518cb1fe2b8015c939010262c7301669c61a91260fb4" exitCode=0 Sep 09 23:16:07.510810 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:16:07.510685 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" event={"ID":"ce8be9ac-786f-4f35-8b40-be4cd36fcb3e","Type":"ContainerDied","Data":"9d885f4a27544bafa5fd518cb1fe2b8015c939010262c7301669c61a91260fb4"} Sep 09 23:16:07.510810 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:16:07.510729 2580 scope.go:117] "RemoveContainer" containerID="eefbe615e947593e2cd611156f2f0451a6cb841e5baf78743b0bb66d4f307c8d" Sep 09 23:16:07.511170 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:16:07.511152 2580 scope.go:117] "RemoveContainer" containerID="9d885f4a27544bafa5fd518cb1fe2b8015c939010262c7301669c61a91260fb4" Sep 09 23:16:07.511361 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:16:07.511341 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x_llm(ce8be9ac-786f-4f35-8b40-be4cd36fcb3e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" podUID="ce8be9ac-786f-4f35-8b40-be4cd36fcb3e" Sep 09 23:16:12.173428 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:16:12.173390 2580 scope.go:117] "RemoveContainer" containerID="c9eac83e1a48b696f860a068f7accca5a4978020e5018effea82ac1d001901c9" Sep 09 23:16:12.173880 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:16:12.173631 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gzhjr_llm(b622d6dd-9145-4fee-94b1-67a9a4695e80)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" podUID="b622d6dd-9145-4fee-94b1-67a9a4695e80" Sep 09 23:16:12.386036 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:16:12.385995 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" Sep 09 23:16:12.386036 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:16:12.386041 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" Sep 09 23:16:12.386250 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:16:12.386053 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" Sep 09 23:16:12.386250 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:16:12.386069 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" Sep 09 23:16:12.386450 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:16:12.386436 2580 scope.go:117] "RemoveContainer" containerID="9d885f4a27544bafa5fd518cb1fe2b8015c939010262c7301669c61a91260fb4" Sep 09 23:16:12.386624 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:16:12.386606 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x_llm(ce8be9ac-786f-4f35-8b40-be4cd36fcb3e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" podUID="ce8be9ac-786f-4f35-8b40-be4cd36fcb3e" Sep 09 23:16:25.174079 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:16:25.174032 2580 scope.go:117] "RemoveContainer" containerID="c9eac83e1a48b696f860a068f7accca5a4978020e5018effea82ac1d001901c9" Sep 09 23:16:25.603102 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:16:25.603067 2580 generic.go:356] "Generic (PLEG): container finished" podID="b622d6dd-9145-4fee-94b1-67a9a4695e80" containerID="3626dd8b6f7d00328fec97166bd35d0892f3761ce3417c9d8a8e1d50e0cb0c05" exitCode=0 Sep 09 23:16:25.603255 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:16:25.603124 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" event={"ID":"b622d6dd-9145-4fee-94b1-67a9a4695e80","Type":"ContainerDied","Data":"3626dd8b6f7d00328fec97166bd35d0892f3761ce3417c9d8a8e1d50e0cb0c05"} Sep 09 23:16:25.603255 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:16:25.603179 2580 scope.go:117] "RemoveContainer" containerID="c9eac83e1a48b696f860a068f7accca5a4978020e5018effea82ac1d001901c9" Sep 09 23:16:25.603557 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:16:25.603526 2580 scope.go:117] "RemoveContainer" containerID="3626dd8b6f7d00328fec97166bd35d0892f3761ce3417c9d8a8e1d50e0cb0c05" Sep 09 23:16:25.603764 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:16:25.603740 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gzhjr_llm(b622d6dd-9145-4fee-94b1-67a9a4695e80)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" podUID="b622d6dd-9145-4fee-94b1-67a9a4695e80" Sep 09 23:16:26.174081 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:16:26.174038 2580 scope.go:117] "RemoveContainer" containerID="9d885f4a27544bafa5fd518cb1fe2b8015c939010262c7301669c61a91260fb4" Sep 09 23:16:26.174451 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:16:26.174198 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x_llm(ce8be9ac-786f-4f35-8b40-be4cd36fcb3e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" podUID="ce8be9ac-786f-4f35-8b40-be4cd36fcb3e" Sep 09 23:16:32.315701 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:16:32.315659 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" Sep 09 23:16:32.316152 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:16:32.315714 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" Sep 09 23:16:32.316152 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:16:32.315732 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" Sep 09 23:16:32.316152 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:16:32.315745 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" Sep 09 23:16:32.316152 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:16:32.316122 2580 scope.go:117] "RemoveContainer" containerID="3626dd8b6f7d00328fec97166bd35d0892f3761ce3417c9d8a8e1d50e0cb0c05" Sep 09 23:16:32.316328 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:16:32.316303 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gzhjr_llm(b622d6dd-9145-4fee-94b1-67a9a4695e80)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" podUID="b622d6dd-9145-4fee-94b1-67a9a4695e80" Sep 09 23:16:38.174404 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:16:38.174363 2580 scope.go:117] "RemoveContainer" containerID="9d885f4a27544bafa5fd518cb1fe2b8015c939010262c7301669c61a91260fb4" Sep 09 23:16:38.174812 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:16:38.174518 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x_llm(ce8be9ac-786f-4f35-8b40-be4cd36fcb3e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" podUID="ce8be9ac-786f-4f35-8b40-be4cd36fcb3e" Sep 09 23:16:43.179560 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:16:43.179533 2580 scope.go:117] "RemoveContainer" containerID="3626dd8b6f7d00328fec97166bd35d0892f3761ce3417c9d8a8e1d50e0cb0c05" Sep 09 23:16:43.179926 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:16:43.179703 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gzhjr_llm(b622d6dd-9145-4fee-94b1-67a9a4695e80)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" podUID="b622d6dd-9145-4fee-94b1-67a9a4695e80" Sep 09 23:16:51.173578 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:16:51.173495 2580 scope.go:117] "RemoveContainer" containerID="9d885f4a27544bafa5fd518cb1fe2b8015c939010262c7301669c61a91260fb4" Sep 09 23:16:51.173942 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:16:51.173663 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x_llm(ce8be9ac-786f-4f35-8b40-be4cd36fcb3e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" podUID="ce8be9ac-786f-4f35-8b40-be4cd36fcb3e" Sep 09 23:16:55.174327 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:16:55.174286 2580 scope.go:117] "RemoveContainer" containerID="3626dd8b6f7d00328fec97166bd35d0892f3761ce3417c9d8a8e1d50e0cb0c05" Sep 09 23:16:55.174726 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:16:55.174484 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gzhjr_llm(b622d6dd-9145-4fee-94b1-67a9a4695e80)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" podUID="b622d6dd-9145-4fee-94b1-67a9a4695e80" Sep 09 23:17:03.173529 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:17:03.173487 2580 scope.go:117] "RemoveContainer" containerID="9d885f4a27544bafa5fd518cb1fe2b8015c939010262c7301669c61a91260fb4" Sep 09 23:17:03.174033 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:17:03.173640 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x_llm(ce8be9ac-786f-4f35-8b40-be4cd36fcb3e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" podUID="ce8be9ac-786f-4f35-8b40-be4cd36fcb3e" Sep 09 23:17:06.174260 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:17:06.174217 2580 scope.go:117] "RemoveContainer" containerID="3626dd8b6f7d00328fec97166bd35d0892f3761ce3417c9d8a8e1d50e0cb0c05" Sep 09 23:17:06.174674 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:17:06.174412 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gzhjr_llm(b622d6dd-9145-4fee-94b1-67a9a4695e80)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" podUID="b622d6dd-9145-4fee-94b1-67a9a4695e80" Sep 09 23:17:14.174373 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:17:14.174341 2580 scope.go:117] "RemoveContainer" containerID="9d885f4a27544bafa5fd518cb1fe2b8015c939010262c7301669c61a91260fb4" Sep 09 23:17:14.174764 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:17:14.174497 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x_llm(ce8be9ac-786f-4f35-8b40-be4cd36fcb3e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" podUID="ce8be9ac-786f-4f35-8b40-be4cd36fcb3e" Sep 09 23:17:20.173700 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:17:20.173664 2580 scope.go:117] "RemoveContainer" containerID="3626dd8b6f7d00328fec97166bd35d0892f3761ce3417c9d8a8e1d50e0cb0c05" Sep 09 23:17:20.174091 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:17:20.173847 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gzhjr_llm(b622d6dd-9145-4fee-94b1-67a9a4695e80)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" podUID="b622d6dd-9145-4fee-94b1-67a9a4695e80" Sep 09 23:17:29.176615 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:17:29.176584 2580 scope.go:117] "RemoveContainer" containerID="9d885f4a27544bafa5fd518cb1fe2b8015c939010262c7301669c61a91260fb4" Sep 09 23:17:29.918010 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:17:29.917954 2580 generic.go:356] "Generic (PLEG): container finished" podID="ce8be9ac-786f-4f35-8b40-be4cd36fcb3e" containerID="59ef168954ec8b86052626a698c86dc8d912918778430d1b690f8abaef125159" exitCode=0 Sep 09 23:17:29.918178 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:17:29.918018 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" event={"ID":"ce8be9ac-786f-4f35-8b40-be4cd36fcb3e","Type":"ContainerDied","Data":"59ef168954ec8b86052626a698c86dc8d912918778430d1b690f8abaef125159"} Sep 09 23:17:29.918178 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:17:29.918054 2580 scope.go:117] "RemoveContainer" containerID="9d885f4a27544bafa5fd518cb1fe2b8015c939010262c7301669c61a91260fb4" Sep 09 23:17:29.918401 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:17:29.918377 2580 scope.go:117] "RemoveContainer" containerID="59ef168954ec8b86052626a698c86dc8d912918778430d1b690f8abaef125159" Sep 09 23:17:29.918595 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:17:29.918572 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x_llm(ce8be9ac-786f-4f35-8b40-be4cd36fcb3e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" podUID="ce8be9ac-786f-4f35-8b40-be4cd36fcb3e" Sep 09 23:17:32.386071 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:17:32.386025 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" Sep 09 23:17:32.386448 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:17:32.386087 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" Sep 09 23:17:32.386448 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:17:32.386100 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" Sep 09 23:17:32.386448 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:17:32.386109 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" Sep 09 23:17:32.386558 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:17:32.386450 2580 scope.go:117] "RemoveContainer" containerID="59ef168954ec8b86052626a698c86dc8d912918778430d1b690f8abaef125159" Sep 09 23:17:32.386625 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:17:32.386605 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x_llm(ce8be9ac-786f-4f35-8b40-be4cd36fcb3e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" podUID="ce8be9ac-786f-4f35-8b40-be4cd36fcb3e" Sep 09 23:17:34.173668 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:17:34.173632 2580 scope.go:117] "RemoveContainer" containerID="3626dd8b6f7d00328fec97166bd35d0892f3761ce3417c9d8a8e1d50e0cb0c05" Sep 09 23:17:34.174149 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:17:34.173790 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gzhjr_llm(b622d6dd-9145-4fee-94b1-67a9a4695e80)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" podUID="b622d6dd-9145-4fee-94b1-67a9a4695e80" Sep 09 23:17:45.173352 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:17:45.173319 2580 scope.go:117] "RemoveContainer" containerID="59ef168954ec8b86052626a698c86dc8d912918778430d1b690f8abaef125159" Sep 09 23:17:45.173736 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:17:45.173484 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x_llm(ce8be9ac-786f-4f35-8b40-be4cd36fcb3e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" podUID="ce8be9ac-786f-4f35-8b40-be4cd36fcb3e" Sep 09 23:17:49.176307 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:17:49.176280 2580 scope.go:117] "RemoveContainer" containerID="3626dd8b6f7d00328fec97166bd35d0892f3761ce3417c9d8a8e1d50e0cb0c05" Sep 09 23:17:49.176676 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:17:49.176477 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gzhjr_llm(b622d6dd-9145-4fee-94b1-67a9a4695e80)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" podUID="b622d6dd-9145-4fee-94b1-67a9a4695e80" Sep 09 23:17:59.177367 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:17:59.177331 2580 scope.go:117] "RemoveContainer" containerID="59ef168954ec8b86052626a698c86dc8d912918778430d1b690f8abaef125159" Sep 09 23:17:59.177831 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:17:59.177537 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x_llm(ce8be9ac-786f-4f35-8b40-be4cd36fcb3e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" podUID="ce8be9ac-786f-4f35-8b40-be4cd36fcb3e" Sep 09 23:18:03.173555 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:03.173514 2580 scope.go:117] "RemoveContainer" containerID="3626dd8b6f7d00328fec97166bd35d0892f3761ce3417c9d8a8e1d50e0cb0c05" Sep 09 23:18:03.173944 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:18:03.173719 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gzhjr_llm(b622d6dd-9145-4fee-94b1-67a9a4695e80)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" podUID="b622d6dd-9145-4fee-94b1-67a9a4695e80" Sep 09 23:18:10.173995 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:10.173953 2580 scope.go:117] "RemoveContainer" containerID="59ef168954ec8b86052626a698c86dc8d912918778430d1b690f8abaef125159" Sep 09 23:18:10.174467 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:18:10.174155 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x_llm(ce8be9ac-786f-4f35-8b40-be4cd36fcb3e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" podUID="ce8be9ac-786f-4f35-8b40-be4cd36fcb3e" Sep 09 23:18:13.211498 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:13.211467 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65d8796fc4-fbltq_594f4ec0-c69c-4b20-98aa-90f2fc9bd6f2/copy-manifests/0.log" Sep 09 23:18:13.218292 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:13.218261 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65d8796fc4-fbltq_594f4ec0-c69c-4b20-98aa-90f2fc9bd6f2/manager/0.log" Sep 09 23:18:13.810809 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:13.810777 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29816595-k9f2g_25f247f6-fb49-4744-8d2d-40a1391049cc/cleanup/0.log" Sep 09 23:18:15.037784 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:15.037758 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-vtq5h_877ad8ce-df51-40f5-9cef-4ed9ec0b8440/kuadrant-console-plugin/0.log" Sep 09 23:18:15.277516 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:15.277482 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-lfr68_89e5537c-524f-4b19-984d-2132c486ba00/limitador/0.log" Sep 09 23:18:16.041420 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:16.041393 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7db58bdd4-57gjk_dba3144e-fd21-40d5-a14e-30533456f538/kube-auth-proxy/0.log" Sep 09 23:18:16.626412 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:16.626364 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-74cbccb686-wxl6p_b7f670bd-13e2-4556-9c0f-ce8317a5ebb5/router/0.log" Sep 09 23:18:17.087601 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:17.087508 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-6685d86975-gzhjr_b622d6dd-9145-4fee-94b1-67a9a4695e80/main/5.log" Sep 09 23:18:17.674434 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:17.674402 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x_ce8be9ac-786f-4f35-8b40-be4cd36fcb3e/main/5.log" Sep 09 23:18:18.173667 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:18.173639 2580 scope.go:117] "RemoveContainer" containerID="3626dd8b6f7d00328fec97166bd35d0892f3761ce3417c9d8a8e1d50e0cb0c05" Sep 09 23:18:18.174070 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:18:18.173806 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gzhjr_llm(b622d6dd-9145-4fee-94b1-67a9a4695e80)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" podUID="b622d6dd-9145-4fee-94b1-67a9a4695e80" Sep 09 23:18:21.174360 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:21.174280 2580 scope.go:117] "RemoveContainer" containerID="59ef168954ec8b86052626a698c86dc8d912918778430d1b690f8abaef125159" Sep 09 23:18:21.174719 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:18:21.174451 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x_llm(ce8be9ac-786f-4f35-8b40-be4cd36fcb3e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" podUID="ce8be9ac-786f-4f35-8b40-be4cd36fcb3e" Sep 09 23:18:22.744534 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:22.744494 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5vzlx/must-gather-czmkz"] Sep 09 23:18:22.745055 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:22.745037 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25f247f6-fb49-4744-8d2d-40a1391049cc" containerName="cleanup" Sep 09 23:18:22.745135 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:22.745060 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="25f247f6-fb49-4744-8d2d-40a1391049cc" containerName="cleanup" Sep 09 23:18:22.745265 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:22.745251 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="25f247f6-fb49-4744-8d2d-40a1391049cc" containerName="cleanup" Sep 09 23:18:22.748724 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:22.748702 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5vzlx/must-gather-czmkz" Sep 09 23:18:22.751413 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:22.751385 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5vzlx\"/\"kube-root-ca.crt\"" Sep 09 23:18:22.751537 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:22.751429 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5vzlx\"/\"openshift-service-ca.crt\"" Sep 09 23:18:22.752648 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:22.752630 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5vzlx\"/\"default-dockercfg-tghjq\"" Sep 09 23:18:22.759512 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:22.759486 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5vzlx/must-gather-czmkz"] Sep 09 23:18:22.862637 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:22.862587 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29xtd\" (UniqueName: \"kubernetes.io/projected/b56c0a68-ee49-4ee9-a4be-9967e512d0d0-kube-api-access-29xtd\") pod \"must-gather-czmkz\" (UID: \"b56c0a68-ee49-4ee9-a4be-9967e512d0d0\") " pod="openshift-must-gather-5vzlx/must-gather-czmkz" Sep 09 23:18:22.862637 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:22.862652 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b56c0a68-ee49-4ee9-a4be-9967e512d0d0-must-gather-output\") pod \"must-gather-czmkz\" (UID: \"b56c0a68-ee49-4ee9-a4be-9967e512d0d0\") " pod="openshift-must-gather-5vzlx/must-gather-czmkz" Sep 09 23:18:22.963531 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:22.963482 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29xtd\" (UniqueName: \"kubernetes.io/projected/b56c0a68-ee49-4ee9-a4be-9967e512d0d0-kube-api-access-29xtd\") pod \"must-gather-czmkz\" (UID: \"b56c0a68-ee49-4ee9-a4be-9967e512d0d0\") " pod="openshift-must-gather-5vzlx/must-gather-czmkz" Sep 09 23:18:22.963703 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:22.963547 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b56c0a68-ee49-4ee9-a4be-9967e512d0d0-must-gather-output\") pod \"must-gather-czmkz\" (UID: \"b56c0a68-ee49-4ee9-a4be-9967e512d0d0\") " pod="openshift-must-gather-5vzlx/must-gather-czmkz" Sep 09 23:18:22.963879 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:22.963859 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b56c0a68-ee49-4ee9-a4be-9967e512d0d0-must-gather-output\") pod \"must-gather-czmkz\" (UID: \"b56c0a68-ee49-4ee9-a4be-9967e512d0d0\") " pod="openshift-must-gather-5vzlx/must-gather-czmkz" Sep 09 23:18:22.971999 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:22.971937 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29xtd\" (UniqueName: \"kubernetes.io/projected/b56c0a68-ee49-4ee9-a4be-9967e512d0d0-kube-api-access-29xtd\") pod \"must-gather-czmkz\" (UID: \"b56c0a68-ee49-4ee9-a4be-9967e512d0d0\") " pod="openshift-must-gather-5vzlx/must-gather-czmkz" Sep 09 23:18:23.058474 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:23.058376 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5vzlx/must-gather-czmkz" Sep 09 23:18:23.340633 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:23.340602 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5vzlx/must-gather-czmkz"] Sep 09 23:18:23.342336 ip-10-0-131-92 kubenswrapper[2580]: W0909 23:18:23.342304 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb56c0a68_ee49_4ee9_a4be_9967e512d0d0.slice/crio-951e552da1df201d55803d10d80917f95bc0e99c614ddb64b002d8d324f78f8a WatchSource:0}: Error finding container 951e552da1df201d55803d10d80917f95bc0e99c614ddb64b002d8d324f78f8a: Status 404 returned error can't find the container with id 951e552da1df201d55803d10d80917f95bc0e99c614ddb64b002d8d324f78f8a Sep 09 23:18:24.178135 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:24.178106 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5vzlx/must-gather-czmkz" event={"ID":"b56c0a68-ee49-4ee9-a4be-9967e512d0d0","Type":"ContainerStarted","Data":"951e552da1df201d55803d10d80917f95bc0e99c614ddb64b002d8d324f78f8a"} Sep 09 23:18:25.187258 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:25.187218 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5vzlx/must-gather-czmkz" event={"ID":"b56c0a68-ee49-4ee9-a4be-9967e512d0d0","Type":"ContainerStarted","Data":"6e819c95218f234138923023c9e0f14d1ef3e4eef9455b2524492002a07565b3"} Sep 09 23:18:25.187258 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:25.187266 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5vzlx/must-gather-czmkz" event={"ID":"b56c0a68-ee49-4ee9-a4be-9967e512d0d0","Type":"ContainerStarted","Data":"ef7c0c2f976e92072d66954e4f5b6410757b894ab4316fdf11f6e19dc8490941"} Sep 09 23:18:25.204658 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:25.204605 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5vzlx/must-gather-czmkz" podStartSLOduration=2.380600662 podStartE2EDuration="3.204588391s" totalImagesPullingTime="823.987729ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 23:18:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:18:25.20415452 +0000 UTC m=+1166.554021582" watchObservedRunningTime="2026-09-09 23:18:25.204588391 +0000 UTC m=+1166.554455424" Sep 09 23:18:25.757919 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:25.757878 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-b2gxt_66786687-e3fe-42fd-8199-7f2a95232aa8/global-pull-secret-syncer/0.log" Sep 09 23:18:25.921740 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:25.921707 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ps2b2_2a2906d3-5dc1-441e-b7f8-a97e2a67d4ed/konnectivity-agent/0.log" Sep 09 23:18:25.986294 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:25.986263 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-92.ec2.internal_8d896ae2507ca9bca5f4fea55fdadcb0/haproxy/0.log" Sep 09 23:18:30.497560 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:30.497459 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-vtq5h_877ad8ce-df51-40f5-9cef-4ed9ec0b8440/kuadrant-console-plugin/0.log" Sep 09 23:18:30.598493 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:30.598446 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-lfr68_89e5537c-524f-4b19-984d-2132c486ba00/limitador/0.log" Sep 09 23:18:31.183619 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:31.183582 2580 scope.go:117] "RemoveContainer" containerID="3626dd8b6f7d00328fec97166bd35d0892f3761ce3417c9d8a8e1d50e0cb0c05" Sep 09 23:18:31.184134 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:18:31.184110 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gzhjr_llm(b622d6dd-9145-4fee-94b1-67a9a4695e80)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" podUID="b622d6dd-9145-4fee-94b1-67a9a4695e80" Sep 09 23:18:32.417226 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:32.417118 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-gbt9m_caacaea7-7d91-480b-b93a-e50c9a7f1d25/kube-state-metrics/0.log" Sep 09 23:18:32.437260 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:32.437188 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-gbt9m_caacaea7-7d91-480b-b93a-e50c9a7f1d25/kube-rbac-proxy-main/0.log" Sep 09 23:18:32.460597 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:32.460560 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-gbt9m_caacaea7-7d91-480b-b93a-e50c9a7f1d25/kube-rbac-proxy-self/0.log" Sep 09 23:18:32.748241 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:32.748202 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wqs6f_e2231319-641c-40bb-901b-2577df536ab0/node-exporter/0.log" Sep 09 23:18:32.772355 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:32.772285 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wqs6f_e2231319-641c-40bb-901b-2577df536ab0/kube-rbac-proxy/0.log" Sep 09 23:18:32.796987 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:32.796948 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wqs6f_e2231319-641c-40bb-901b-2577df536ab0/init-textfile/0.log" Sep 09 23:18:32.826453 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:32.826420 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-nw5xf_92e597ee-8c57-4c77-908e-6e8045ffcded/kube-rbac-proxy-main/0.log" Sep 09 23:18:32.853741 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:32.853662 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-nw5xf_92e597ee-8c57-4c77-908e-6e8045ffcded/kube-rbac-proxy-self/0.log" Sep 09 23:18:32.876541 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:32.876467 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-nw5xf_92e597ee-8c57-4c77-908e-6e8045ffcded/openshift-state-metrics/0.log" Sep 09 23:18:32.932091 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:32.932044 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2554a324-4a9f-46b0-bb48-099ba2dfe771/prometheus/0.log" Sep 09 23:18:32.955080 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:32.955040 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2554a324-4a9f-46b0-bb48-099ba2dfe771/config-reloader/0.log" Sep 09 23:18:32.981795 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:32.981752 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2554a324-4a9f-46b0-bb48-099ba2dfe771/thanos-sidecar/0.log" Sep 09 23:18:33.006014 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:33.005968 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2554a324-4a9f-46b0-bb48-099ba2dfe771/kube-rbac-proxy-web/0.log" Sep 09 23:18:33.042940 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:33.042905 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2554a324-4a9f-46b0-bb48-099ba2dfe771/kube-rbac-proxy/0.log" Sep 09 23:18:33.078273 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:33.078232 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2554a324-4a9f-46b0-bb48-099ba2dfe771/kube-rbac-proxy-thanos/0.log" Sep 09 23:18:33.109600 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:33.109513 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2554a324-4a9f-46b0-bb48-099ba2dfe771/init-config-reloader/0.log" Sep 09 23:18:33.141444 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:33.141409 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-m7l6j_32c1d310-58b7-48ce-a72e-bd2dd4b64ab4/prometheus-operator/0.log" Sep 09 23:18:33.166270 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:33.166235 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-m7l6j_32c1d310-58b7-48ce-a72e-bd2dd4b64ab4/kube-rbac-proxy/0.log" Sep 09 23:18:33.199831 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:33.199789 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-zpssz_275dc765-1345-4219-a0b6-e12119073feb/prometheus-operator-admission-webhook/0.log" Sep 09 23:18:33.330711 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:33.330680 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f85c99747-bcwgk_7106047f-cae5-4ae2-8de3-1b71715187a1/thanos-query/0.log" Sep 09 23:18:33.352468 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:33.352436 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f85c99747-bcwgk_7106047f-cae5-4ae2-8de3-1b71715187a1/kube-rbac-proxy-web/0.log" Sep 09 23:18:33.373499 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:33.373472 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f85c99747-bcwgk_7106047f-cae5-4ae2-8de3-1b71715187a1/kube-rbac-proxy/0.log" Sep 09 23:18:33.396703 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:33.396669 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f85c99747-bcwgk_7106047f-cae5-4ae2-8de3-1b71715187a1/prom-label-proxy/0.log" Sep 09 23:18:33.439217 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:33.439182 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f85c99747-bcwgk_7106047f-cae5-4ae2-8de3-1b71715187a1/kube-rbac-proxy-rules/0.log" Sep 09 23:18:33.462329 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:33.462295 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f85c99747-bcwgk_7106047f-cae5-4ae2-8de3-1b71715187a1/kube-rbac-proxy-metrics/0.log" Sep 09 23:18:34.916117 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:34.916084 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-zmvn9_5f001595-27ff-43d7-94d2-7ddc18d99671/networking-console-plugin/0.log" Sep 09 23:18:35.563372 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:35.563327 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xht2x_3b6dd849-6a7e-4141-a460-e9ee0e7ff07c/console-operator/2.log" Sep 09 23:18:35.572715 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:35.572680 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-xht2x_3b6dd849-6a7e-4141-a460-e9ee0e7ff07c/console-operator/3.log" Sep 09 23:18:36.174160 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:36.174135 2580 scope.go:117] "RemoveContainer" containerID="59ef168954ec8b86052626a698c86dc8d912918778430d1b690f8abaef125159" Sep 09 23:18:36.174764 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:18:36.174728 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x_llm(ce8be9ac-786f-4f35-8b40-be4cd36fcb3e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" podUID="ce8be9ac-786f-4f35-8b40-be4cd36fcb3e" Sep 09 23:18:36.175551 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:36.175528 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5vzlx/perf-node-gather-daemonset-6p9rz"] Sep 09 23:18:36.181397 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:36.181366 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5vzlx/perf-node-gather-daemonset-6p9rz" Sep 09 23:18:36.188438 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:36.188366 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5vzlx/perf-node-gather-daemonset-6p9rz"] Sep 09 23:18:36.306267 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:36.306223 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/902ba15c-2f6a-4182-a9ae-defff2f23a83-sys\") pod \"perf-node-gather-daemonset-6p9rz\" (UID: \"902ba15c-2f6a-4182-a9ae-defff2f23a83\") " pod="openshift-must-gather-5vzlx/perf-node-gather-daemonset-6p9rz" Sep 09 23:18:36.306861 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:36.306841 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/902ba15c-2f6a-4182-a9ae-defff2f23a83-proc\") pod \"perf-node-gather-daemonset-6p9rz\" (UID: \"902ba15c-2f6a-4182-a9ae-defff2f23a83\") " pod="openshift-must-gather-5vzlx/perf-node-gather-daemonset-6p9rz" Sep 09 23:18:36.307152 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:36.307135 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/902ba15c-2f6a-4182-a9ae-defff2f23a83-podres\") pod \"perf-node-gather-daemonset-6p9rz\" (UID: \"902ba15c-2f6a-4182-a9ae-defff2f23a83\") " pod="openshift-must-gather-5vzlx/perf-node-gather-daemonset-6p9rz" Sep 09 23:18:36.307308 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:36.307293 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/902ba15c-2f6a-4182-a9ae-defff2f23a83-lib-modules\") pod \"perf-node-gather-daemonset-6p9rz\" (UID: \"902ba15c-2f6a-4182-a9ae-defff2f23a83\") " pod="openshift-must-gather-5vzlx/perf-node-gather-daemonset-6p9rz" Sep 09 23:18:36.307408 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:36.307394 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxg5g\" (UniqueName: \"kubernetes.io/projected/902ba15c-2f6a-4182-a9ae-defff2f23a83-kube-api-access-dxg5g\") pod \"perf-node-gather-daemonset-6p9rz\" (UID: \"902ba15c-2f6a-4182-a9ae-defff2f23a83\") " pod="openshift-must-gather-5vzlx/perf-node-gather-daemonset-6p9rz" Sep 09 23:18:36.410595 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:36.410110 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/902ba15c-2f6a-4182-a9ae-defff2f23a83-proc\") pod \"perf-node-gather-daemonset-6p9rz\" (UID: \"902ba15c-2f6a-4182-a9ae-defff2f23a83\") " pod="openshift-must-gather-5vzlx/perf-node-gather-daemonset-6p9rz" Sep 09 23:18:36.410595 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:36.410220 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/902ba15c-2f6a-4182-a9ae-defff2f23a83-podres\") pod \"perf-node-gather-daemonset-6p9rz\" (UID: \"902ba15c-2f6a-4182-a9ae-defff2f23a83\") " pod="openshift-must-gather-5vzlx/perf-node-gather-daemonset-6p9rz" Sep 09 23:18:36.410595 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:36.410253 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/902ba15c-2f6a-4182-a9ae-defff2f23a83-lib-modules\") pod \"perf-node-gather-daemonset-6p9rz\" (UID: \"902ba15c-2f6a-4182-a9ae-defff2f23a83\") " pod="openshift-must-gather-5vzlx/perf-node-gather-daemonset-6p9rz" Sep 09 23:18:36.410595 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:36.410275 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxg5g\" (UniqueName: \"kubernetes.io/projected/902ba15c-2f6a-4182-a9ae-defff2f23a83-kube-api-access-dxg5g\") pod \"perf-node-gather-daemonset-6p9rz\" (UID: \"902ba15c-2f6a-4182-a9ae-defff2f23a83\") " pod="openshift-must-gather-5vzlx/perf-node-gather-daemonset-6p9rz" Sep 09 23:18:36.410595 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:36.410344 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/902ba15c-2f6a-4182-a9ae-defff2f23a83-sys\") pod \"perf-node-gather-daemonset-6p9rz\" (UID: \"902ba15c-2f6a-4182-a9ae-defff2f23a83\") " pod="openshift-must-gather-5vzlx/perf-node-gather-daemonset-6p9rz" Sep 09 23:18:36.410595 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:36.410463 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/902ba15c-2f6a-4182-a9ae-defff2f23a83-sys\") pod \"perf-node-gather-daemonset-6p9rz\" (UID: \"902ba15c-2f6a-4182-a9ae-defff2f23a83\") " pod="openshift-must-gather-5vzlx/perf-node-gather-daemonset-6p9rz" Sep 09 23:18:36.410595 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:36.410531 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/902ba15c-2f6a-4182-a9ae-defff2f23a83-proc\") pod \"perf-node-gather-daemonset-6p9rz\" (UID: \"902ba15c-2f6a-4182-a9ae-defff2f23a83\") " pod="openshift-must-gather-5vzlx/perf-node-gather-daemonset-6p9rz" Sep 09 23:18:36.411095 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:36.410633 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/902ba15c-2f6a-4182-a9ae-defff2f23a83-podres\") pod \"perf-node-gather-daemonset-6p9rz\" (UID: \"902ba15c-2f6a-4182-a9ae-defff2f23a83\") " pod="openshift-must-gather-5vzlx/perf-node-gather-daemonset-6p9rz" Sep 09 23:18:36.411095 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:36.410707 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/902ba15c-2f6a-4182-a9ae-defff2f23a83-lib-modules\") pod \"perf-node-gather-daemonset-6p9rz\" (UID: \"902ba15c-2f6a-4182-a9ae-defff2f23a83\") " pod="openshift-must-gather-5vzlx/perf-node-gather-daemonset-6p9rz" Sep 09 23:18:36.418783 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:36.418751 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxg5g\" (UniqueName: \"kubernetes.io/projected/902ba15c-2f6a-4182-a9ae-defff2f23a83-kube-api-access-dxg5g\") pod \"perf-node-gather-daemonset-6p9rz\" (UID: \"902ba15c-2f6a-4182-a9ae-defff2f23a83\") " pod="openshift-must-gather-5vzlx/perf-node-gather-daemonset-6p9rz" Sep 09 23:18:36.498045 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:36.497934 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5vzlx/perf-node-gather-daemonset-6p9rz" Sep 09 23:18:36.792068 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:36.791301 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5vzlx/perf-node-gather-daemonset-6p9rz"] Sep 09 23:18:37.296479 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:37.296437 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5vzlx/perf-node-gather-daemonset-6p9rz" event={"ID":"902ba15c-2f6a-4182-a9ae-defff2f23a83","Type":"ContainerStarted","Data":"6221627ba456463496ebbe6edd46b956b6863c16eb330e9ced8266ed022973b8"} Sep 09 23:18:37.296893 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:37.296504 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5vzlx/perf-node-gather-daemonset-6p9rz" event={"ID":"902ba15c-2f6a-4182-a9ae-defff2f23a83","Type":"ContainerStarted","Data":"8a7b3f24f70a98c188d993aed58169728bfffd8fd3307fe7881b1be135060feb"} Sep 09 23:18:37.296893 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:37.296534 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5vzlx/perf-node-gather-daemonset-6p9rz" Sep 09 23:18:38.070206 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:38.070172 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-b6zvm_6251fed2-5675-42f5-b0d1-788e1a894092/dns/0.log" Sep 09 23:18:38.091859 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:38.091832 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-b6zvm_6251fed2-5675-42f5-b0d1-788e1a894092/kube-rbac-proxy/0.log" Sep 09 23:18:38.275804 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:38.275773 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lwqr2_dadd077d-e65f-49c0-a1bf-b8299ef2bb89/dns-node-resolver/0.log" Sep 09 23:18:38.854294 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:38.854268 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jx58q_0fe9b158-c1b2-498d-aecc-d32aee3c51c6/node-ca/0.log" Sep 09 23:18:39.860041 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:39.860005 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7db58bdd4-57gjk_dba3144e-fd21-40d5-a14e-30533456f538/kube-auth-proxy/0.log" Sep 09 23:18:39.995461 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:39.995423 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-74cbccb686-wxl6p_b7f670bd-13e2-4556-9c0f-ce8317a5ebb5/router/0.log" Sep 09 23:18:40.603037 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:40.602968 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5dl4c_d1a033d1-6eb8-472d-95dc-5a1c26b60867/serve-healthcheck-canary/0.log" Sep 09 23:18:41.161823 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:41.161781 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-86mkf_7e149a18-7de7-43d3-89ca-14e842fa6d77/kube-rbac-proxy/0.log" Sep 09 23:18:41.182515 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:41.182485 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-86mkf_7e149a18-7de7-43d3-89ca-14e842fa6d77/exporter/0.log" Sep 09 23:18:41.203630 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:41.203587 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-86mkf_7e149a18-7de7-43d3-89ca-14e842fa6d77/extractor/0.log" Sep 09 23:18:43.318567 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:43.318537 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5vzlx/perf-node-gather-daemonset-6p9rz" Sep 09 23:18:43.335276 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:43.335210 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5vzlx/perf-node-gather-daemonset-6p9rz" podStartSLOduration=7.33519366 podStartE2EDuration="7.33519366s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 23:18:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 23:18:37.316307832 +0000 UTC m=+1178.666174867" watchObservedRunningTime="2026-09-09 23:18:43.33519366 +0000 UTC m=+1184.685060695" Sep 09 23:18:43.718409 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:43.718372 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65d8796fc4-fbltq_594f4ec0-c69c-4b20-98aa-90f2fc9bd6f2/manager/0.log" Sep 09 23:18:43.737335 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:43.737305 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65d8796fc4-fbltq_594f4ec0-c69c-4b20-98aa-90f2fc9bd6f2/copy-manifests/0.log" Sep 09 23:18:44.173455 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:44.173416 2580 scope.go:117] "RemoveContainer" containerID="3626dd8b6f7d00328fec97166bd35d0892f3761ce3417c9d8a8e1d50e0cb0c05" Sep 09 23:18:44.173682 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:18:44.173657 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-gzhjr_llm(b622d6dd-9145-4fee-94b1-67a9a4695e80)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-gzhjr" podUID="b622d6dd-9145-4fee-94b1-67a9a4695e80" Sep 09 23:18:45.032307 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:45.032273 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5c9946bf76-wwkft_d3677ab8-223a-4bb5-bebf-56e5a37b6f37/manager/0.log" Sep 09 23:18:45.060261 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:45.060224 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-hn2sz_10251ad7-f639-4f74-a01e-472453daadbc/openshift-lws-operator/0.log" Sep 09 23:18:46.569269 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:46.569231 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7db58bdd4-57gjk_dba3144e-fd21-40d5-a14e-30533456f538/kube-auth-proxy/0.log" Sep 09 23:18:46.646178 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:46.646144 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-74cbccb686-wxl6p_b7f670bd-13e2-4556-9c0f-ce8317a5ebb5/router/0.log" Sep 09 23:18:47.869357 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:47.869275 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7db58bdd4-57gjk_dba3144e-fd21-40d5-a14e-30533456f538/kube-auth-proxy/0.log" Sep 09 23:18:47.972324 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:47.972274 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-74cbccb686-wxl6p_b7f670bd-13e2-4556-9c0f-ce8317a5ebb5/router/0.log" Sep 09 23:18:48.173543 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:48.173516 2580 scope.go:117] "RemoveContainer" containerID="59ef168954ec8b86052626a698c86dc8d912918778430d1b690f8abaef125159" Sep 09 23:18:48.173758 ip-10-0-131-92 kubenswrapper[2580]: E0909 23:18:48.173694 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x_llm(ce8be9ac-786f-4f35-8b40-be4cd36fcb3e)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-w7z8x" podUID="ce8be9ac-786f-4f35-8b40-be4cd36fcb3e" Sep 09 23:18:50.710098 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:50.710053 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-dtbbq_4ebd018d-e004-4bd2-94ed-594b76e747ca/kube-storage-version-migrator-operator/1.log" Sep 09 23:18:50.711854 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:50.711827 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-dtbbq_4ebd018d-e004-4bd2-94ed-594b76e747ca/kube-storage-version-migrator-operator/0.log" Sep 09 23:18:51.712325 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:51.712294 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dc58c_09c46a31-75e9-40a5-856f-4f53ae4b0a3b/kube-multus-additional-cni-plugins/0.log" Sep 09 23:18:51.732266 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:51.732236 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dc58c_09c46a31-75e9-40a5-856f-4f53ae4b0a3b/egress-router-binary-copy/0.log" Sep 09 23:18:51.752805 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:51.752771 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dc58c_09c46a31-75e9-40a5-856f-4f53ae4b0a3b/cni-plugins/0.log" Sep 09 23:18:51.772995 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:51.772944 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dc58c_09c46a31-75e9-40a5-856f-4f53ae4b0a3b/bond-cni-plugin/0.log" Sep 09 23:18:51.793368 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:51.793335 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dc58c_09c46a31-75e9-40a5-856f-4f53ae4b0a3b/routeoverride-cni/0.log" Sep 09 23:18:51.813579 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:51.813550 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dc58c_09c46a31-75e9-40a5-856f-4f53ae4b0a3b/whereabouts-cni-bincopy/0.log" Sep 09 23:18:51.834341 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:51.834309 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dc58c_09c46a31-75e9-40a5-856f-4f53ae4b0a3b/whereabouts-cni/0.log" Sep 09 23:18:52.262239 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:52.262204 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-w9lv9_dcc18d79-d518-4d89-ac61-0e3e1ff55794/kube-multus/0.log" Sep 09 23:18:52.286529 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:52.286500 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-8dwkd_7cbe663c-0531-4339-b48b-c4d4c3dd5c2b/network-metrics-daemon/0.log" Sep 09 23:18:52.348548 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:52.348517 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-8dwkd_7cbe663c-0531-4339-b48b-c4d4c3dd5c2b/kube-rbac-proxy/0.log" Sep 09 23:18:53.721304 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:53.721268 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-prbnz_bece1799-8601-448b-9fb5-67fe028e3d7e/ovn-controller/0.log" Sep 09 23:18:53.738835 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:53.738797 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-prbnz_bece1799-8601-448b-9fb5-67fe028e3d7e/ovn-acl-logging/0.log" Sep 09 23:18:53.766536 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:53.766500 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-prbnz_bece1799-8601-448b-9fb5-67fe028e3d7e/kube-rbac-proxy-node/0.log" Sep 09 23:18:53.789705 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:53.789668 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-prbnz_bece1799-8601-448b-9fb5-67fe028e3d7e/kube-rbac-proxy-ovn-metrics/0.log" Sep 09 23:18:53.807239 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:53.807211 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-prbnz_bece1799-8601-448b-9fb5-67fe028e3d7e/northd/0.log" Sep 09 23:18:53.827207 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:53.827145 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-prbnz_bece1799-8601-448b-9fb5-67fe028e3d7e/nbdb/0.log" Sep 09 23:18:53.847302 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:53.847271 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-prbnz_bece1799-8601-448b-9fb5-67fe028e3d7e/sbdb/0.log" Sep 09 23:18:53.909647 ip-10-0-131-92 kubenswrapper[2580]: I0909 23:18:53.909617 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-prbnz_bece1799-8601-448b-9fb5-67fe028e3d7e/ovnkube-controller/0.log"