Sep 10 03:46:03.650807 ip-10-0-142-77 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 10 03:46:03.650819 ip-10-0-142-77 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 10 03:46:03.650829 ip-10-0-142-77 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 10 03:46:03.651030 ip-10-0-142-77 systemd[1]: Failed to start Kubernetes Kubelet. Sep 10 03:46:13.833331 ip-10-0-142-77 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Sep 10 03:46:13.833353 ip-10-0-142-77 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot c8661f6469664d14bc61c59d5811c289 -- Sep 10 03:48:05.230018 ip-10-0-142-77 systemd[1]: Starting Kubernetes Kubelet... Sep 10 03:48:05.605756 ip-10-0-142-77 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 03:48:05.605756 ip-10-0-142-77 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 10 03:48:05.605756 ip-10-0-142-77 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 03:48:05.605756 ip-10-0-142-77 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 10 03:48:05.605756 ip-10-0-142-77 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 03:48:05.606695 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.606587 2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 10 03:48:05.610068 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610052 2581 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 03:48:05.610068 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610068 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 03:48:05.610127 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610072 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 03:48:05.610127 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610076 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 03:48:05.610127 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610079 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 03:48:05.610127 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610081 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 03:48:05.610127 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610084 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 03:48:05.610127 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610087 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 03:48:05.610127 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610094 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 03:48:05.610127 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610096 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 03:48:05.610127 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610099 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 03:48:05.610127 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610102 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 03:48:05.610127 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610104 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 03:48:05.610127 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610107 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 03:48:05.610127 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610109 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 03:48:05.610127 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610112 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 03:48:05.610127 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610115 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 03:48:05.610127 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610118 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 03:48:05.610127 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610120 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 03:48:05.610127 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610122 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 03:48:05.610127 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610125 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 03:48:05.610579 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610128 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 03:48:05.610579 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610131 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 03:48:05.610579 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610133 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 03:48:05.610579 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610136 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 03:48:05.610579 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610139 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 03:48:05.610579 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610142 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 03:48:05.610579 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610144 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 03:48:05.610579 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610147 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 03:48:05.610579 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610149 2581 feature_gate.go:328] unrecognized feature gate: Example Sep 10 03:48:05.610579 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610152 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 03:48:05.610579 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610154 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 03:48:05.610579 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610157 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 03:48:05.610579 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610159 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 03:48:05.610579 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610162 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 03:48:05.610579 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610166 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 03:48:05.610579 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610168 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 03:48:05.610579 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610170 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 03:48:05.610579 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610173 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 03:48:05.610579 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610176 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 03:48:05.610579 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610178 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 03:48:05.611071 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610180 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 03:48:05.611071 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610183 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 03:48:05.611071 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610185 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 03:48:05.611071 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610188 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 03:48:05.611071 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610190 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 03:48:05.611071 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610193 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 03:48:05.611071 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610197 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 03:48:05.611071 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610201 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 03:48:05.611071 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610204 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 03:48:05.611071 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610207 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 03:48:05.611071 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610210 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 03:48:05.611071 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610212 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 03:48:05.611071 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610215 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 03:48:05.611071 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610219 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 03:48:05.611071 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610222 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 03:48:05.611071 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610224 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 03:48:05.611071 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610226 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 03:48:05.611071 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610229 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 03:48:05.611071 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610231 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 03:48:05.611535 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610234 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 03:48:05.611535 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610239 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 03:48:05.611535 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610242 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 03:48:05.611535 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610244 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 03:48:05.611535 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610247 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 03:48:05.611535 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610249 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 03:48:05.611535 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610252 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 03:48:05.611535 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610254 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 03:48:05.611535 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610256 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 03:48:05.611535 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610259 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 03:48:05.611535 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610261 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 03:48:05.611535 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610264 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 03:48:05.611535 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610267 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 03:48:05.611535 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610270 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 03:48:05.611535 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610272 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 03:48:05.611535 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610274 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 03:48:05.611535 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610277 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 03:48:05.611535 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610280 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 03:48:05.611535 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610282 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 03:48:05.611535 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610284 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 03:48:05.612052 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610287 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 03:48:05.612052 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610289 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 03:48:05.612052 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610292 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 03:48:05.612052 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610294 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 03:48:05.612052 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610296 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 03:48:05.612052 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610299 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 03:48:05.612052 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610303 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 03:48:05.612052 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610306 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 03:48:05.612052 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610309 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 03:48:05.612052 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610311 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 03:48:05.612052 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610314 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 03:48:05.612052 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610317 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 03:48:05.612052 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610320 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 03:48:05.612052 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610322 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 03:48:05.612052 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610325 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 03:48:05.612052 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610327 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 03:48:05.612052 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610330 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 03:48:05.612052 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610332 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 03:48:05.612052 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.610334 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 03:48:05.612500 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611203 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 03:48:05.612500 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611211 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 03:48:05.612500 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611214 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 03:48:05.612500 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611217 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 03:48:05.612500 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611219 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 03:48:05.612500 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611223 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 03:48:05.612500 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611226 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 03:48:05.612500 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611229 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 03:48:05.612500 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611232 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 03:48:05.612500 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611234 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 03:48:05.612500 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611237 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 03:48:05.612500 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611240 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 03:48:05.612500 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611242 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 03:48:05.612500 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611245 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 03:48:05.612500 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611248 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 03:48:05.612500 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611250 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 03:48:05.612500 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611253 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 03:48:05.612500 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611255 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 03:48:05.612500 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611258 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 03:48:05.612500 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611262 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 03:48:05.612997 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611264 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 03:48:05.612997 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611267 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 03:48:05.612997 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611270 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 03:48:05.612997 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611272 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 03:48:05.612997 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611276 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 03:48:05.612997 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611279 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 03:48:05.612997 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611281 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 03:48:05.612997 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611284 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 03:48:05.612997 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611286 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 03:48:05.612997 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611288 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 03:48:05.612997 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611291 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 03:48:05.612997 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611293 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 03:48:05.612997 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611296 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 03:48:05.612997 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611298 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 03:48:05.612997 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611300 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 03:48:05.612997 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611303 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 03:48:05.612997 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611305 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 03:48:05.612997 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611307 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 03:48:05.612997 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611310 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 03:48:05.612997 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611313 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 03:48:05.613471 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611315 2581 feature_gate.go:328] unrecognized feature gate: Example Sep 10 03:48:05.613471 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611318 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 03:48:05.613471 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611320 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 03:48:05.613471 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611323 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 03:48:05.613471 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611326 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 03:48:05.613471 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611328 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 03:48:05.613471 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611330 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 03:48:05.613471 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611333 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 03:48:05.613471 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611337 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 03:48:05.613471 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611341 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 03:48:05.613471 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611344 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 03:48:05.613471 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611347 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 03:48:05.613471 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611350 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 03:48:05.613471 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611352 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 03:48:05.613471 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611355 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 03:48:05.613471 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611358 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 03:48:05.613471 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611360 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 03:48:05.613471 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611363 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 03:48:05.613471 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611365 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 03:48:05.613471 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611368 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 03:48:05.614056 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611372 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 03:48:05.614056 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611375 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 03:48:05.614056 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611379 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 03:48:05.614056 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611382 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 03:48:05.614056 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611385 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 03:48:05.614056 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611388 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 03:48:05.614056 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611392 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 03:48:05.614056 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611395 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 03:48:05.614056 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611398 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 03:48:05.614056 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611401 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 03:48:05.614056 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611404 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 03:48:05.614056 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611407 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 03:48:05.614056 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611409 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 03:48:05.614056 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611412 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 03:48:05.614056 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611414 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 03:48:05.614056 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611417 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 03:48:05.614056 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611419 2581 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 03:48:05.614056 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611422 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 03:48:05.614056 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611424 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 03:48:05.614511 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611427 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 03:48:05.614511 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611429 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 03:48:05.614511 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611432 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 03:48:05.614511 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611434 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 03:48:05.614511 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611437 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 03:48:05.614511 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611440 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 03:48:05.614511 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611443 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 03:48:05.614511 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611445 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 03:48:05.614511 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611448 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 03:48:05.614511 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611450 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 03:48:05.614511 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611453 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 03:48:05.614511 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611455 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 03:48:05.614511 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611457 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 03:48:05.614511 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611460 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 03:48:05.614511 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611462 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 03:48:05.614511 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611465 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 03:48:05.614511 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611467 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 03:48:05.614511 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611470 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 03:48:05.614511 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611472 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 03:48:05.614511 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611474 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 03:48:05.615001 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611895 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 03:48:05.615001 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611900 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 03:48:05.615001 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611903 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 03:48:05.615001 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611906 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 03:48:05.615001 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611914 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 03:48:05.615001 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611917 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 03:48:05.615001 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611920 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 03:48:05.615001 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611923 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 03:48:05.615001 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611925 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 03:48:05.615001 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611928 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 03:48:05.615001 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611930 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 03:48:05.615001 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611933 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 03:48:05.615001 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611935 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 03:48:05.615001 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611937 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 03:48:05.615001 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611940 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 03:48:05.615001 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611943 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 03:48:05.615001 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611948 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 03:48:05.615001 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611951 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 03:48:05.615001 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611953 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 03:48:05.615466 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611957 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 03:48:05.615466 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611960 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 03:48:05.615466 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611963 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 03:48:05.615466 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611965 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 03:48:05.615466 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611968 2581 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 03:48:05.615466 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611971 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 03:48:05.615466 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611974 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 03:48:05.615466 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611976 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 03:48:05.615466 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611979 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 03:48:05.615466 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611981 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 03:48:05.615466 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611984 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 03:48:05.615466 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611986 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 03:48:05.615466 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611989 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 03:48:05.615466 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611992 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 03:48:05.615466 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611994 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 03:48:05.615466 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611996 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 03:48:05.615466 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.611999 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 03:48:05.615466 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612001 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 03:48:05.615466 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612009 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 03:48:05.615466 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612012 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 03:48:05.615974 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612014 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 03:48:05.615974 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612017 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 03:48:05.615974 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612019 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 03:48:05.615974 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612021 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 03:48:05.615974 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612024 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 03:48:05.615974 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612026 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 03:48:05.615974 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612029 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 03:48:05.615974 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612031 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 03:48:05.615974 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612036 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 03:48:05.615974 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612040 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 03:48:05.615974 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612042 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 03:48:05.615974 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612045 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 03:48:05.615974 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612047 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 03:48:05.615974 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612050 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 03:48:05.615974 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612052 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 03:48:05.615974 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612055 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 03:48:05.615974 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612058 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 03:48:05.615974 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612060 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 03:48:05.615974 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612063 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 03:48:05.615974 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612066 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 03:48:05.616453 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612068 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 03:48:05.616453 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612071 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 03:48:05.616453 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612074 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 03:48:05.616453 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612076 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 03:48:05.616453 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612080 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 03:48:05.616453 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612082 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 03:48:05.616453 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612085 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 03:48:05.616453 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612087 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 03:48:05.616453 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612090 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 03:48:05.616453 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612092 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 03:48:05.616453 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612095 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 03:48:05.616453 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612098 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 03:48:05.616453 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612100 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 03:48:05.616453 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612103 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 03:48:05.616453 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612105 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 03:48:05.616453 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612108 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 03:48:05.616453 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612110 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 03:48:05.616453 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612113 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 03:48:05.616453 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612115 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 03:48:05.616453 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612118 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 03:48:05.617002 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612122 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 03:48:05.617002 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612128 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 03:48:05.617002 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612131 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 03:48:05.617002 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612134 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 03:48:05.617002 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612137 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 03:48:05.617002 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612139 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 03:48:05.617002 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612142 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 03:48:05.617002 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612146 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 03:48:05.617002 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612149 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 03:48:05.617002 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612152 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 03:48:05.617002 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612156 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 03:48:05.617002 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612159 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 03:48:05.617002 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612162 2581 feature_gate.go:328] unrecognized feature gate: Example Sep 10 03:48:05.617002 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612164 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 03:48:05.617002 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612167 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 03:48:05.617002 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612170 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 03:48:05.617002 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612172 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 03:48:05.617002 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612175 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 03:48:05.617002 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612177 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 03:48:05.617449 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612180 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 03:48:05.617449 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612257 2581 flags.go:64] FLAG: --address="0.0.0.0" Sep 10 03:48:05.617449 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612294 2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 10 03:48:05.617449 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612301 2581 flags.go:64] FLAG: --anonymous-auth="true" Sep 10 03:48:05.617449 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612305 2581 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 10 03:48:05.617449 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612311 2581 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 10 03:48:05.617449 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612314 2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 10 03:48:05.617449 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612318 2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 10 03:48:05.617449 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612322 2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 10 03:48:05.617449 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612326 2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 10 03:48:05.617449 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612328 2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 10 03:48:05.617449 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612332 2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 10 03:48:05.617449 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612335 2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 10 03:48:05.617449 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612340 2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 10 03:48:05.617449 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612344 2581 flags.go:64] FLAG: --cgroup-root="" Sep 10 03:48:05.617449 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612347 2581 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 10 03:48:05.617449 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612350 2581 flags.go:64] FLAG: --client-ca-file="" Sep 10 03:48:05.617449 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612352 2581 flags.go:64] FLAG: --cloud-provider="external" Sep 10 03:48:05.617449 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612355 2581 flags.go:64] FLAG: --cluster-dns="[]" Sep 10 03:48:05.617449 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612359 2581 flags.go:64] FLAG: --cluster-domain="" Sep 10 03:48:05.617449 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612362 2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 10 03:48:05.617449 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612365 2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 10 03:48:05.617449 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612368 2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 10 03:48:05.617449 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612371 2581 flags.go:64] FLAG: --container-log-max-files="5" Sep 10 03:48:05.618016 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612375 2581 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 10 03:48:05.618016 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612378 2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 10 03:48:05.618016 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612381 2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 10 03:48:05.618016 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612384 2581 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 10 03:48:05.618016 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612387 2581 flags.go:64] FLAG: --contention-profiling="false" Sep 10 03:48:05.618016 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612390 2581 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 10 03:48:05.618016 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612392 2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 10 03:48:05.618016 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612395 2581 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 10 03:48:05.618016 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612398 2581 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 10 03:48:05.618016 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612402 2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 10 03:48:05.618016 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612405 2581 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 10 03:48:05.618016 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612408 2581 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 10 03:48:05.618016 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612411 2581 flags.go:64] FLAG: --enable-load-reader="false" Sep 10 03:48:05.618016 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612413 2581 flags.go:64] FLAG: --enable-server="true" Sep 10 03:48:05.618016 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612417 2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 10 03:48:05.618016 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612421 2581 flags.go:64] FLAG: --event-burst="100" Sep 10 03:48:05.618016 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612424 2581 flags.go:64] FLAG: --event-qps="50" Sep 10 03:48:05.618016 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612427 2581 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 10 03:48:05.618016 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612430 2581 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 10 03:48:05.618016 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612433 2581 flags.go:64] FLAG: --eviction-hard="" Sep 10 03:48:05.618016 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612436 2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 10 03:48:05.618016 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612440 2581 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 10 03:48:05.618016 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612444 2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 10 03:48:05.618016 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612447 2581 flags.go:64] FLAG: --eviction-soft="" Sep 10 03:48:05.618016 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612450 2581 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 10 03:48:05.618588 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612453 2581 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 10 03:48:05.618588 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612455 2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 10 03:48:05.618588 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612458 2581 flags.go:64] FLAG: --experimental-mounter-path="" Sep 10 03:48:05.618588 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612461 2581 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 10 03:48:05.618588 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612464 2581 flags.go:64] FLAG: --fail-swap-on="true" Sep 10 03:48:05.618588 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612467 2581 flags.go:64] FLAG: --feature-gates="" Sep 10 03:48:05.618588 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612471 2581 flags.go:64] FLAG: --file-check-frequency="20s" Sep 10 03:48:05.618588 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612474 2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 10 03:48:05.618588 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612476 2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 10 03:48:05.618588 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612479 2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 10 03:48:05.618588 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612482 2581 flags.go:64] FLAG: --healthz-port="10248" Sep 10 03:48:05.618588 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612485 2581 flags.go:64] FLAG: --help="false" Sep 10 03:48:05.618588 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612488 2581 flags.go:64] FLAG: --hostname-override="ip-10-0-142-77.ec2.internal" Sep 10 03:48:05.618588 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612491 2581 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 10 03:48:05.618588 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612494 2581 flags.go:64] FLAG: --http-check-frequency="20s" Sep 10 03:48:05.618588 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612496 2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 10 03:48:05.618588 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612499 2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 10 03:48:05.618588 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612503 2581 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 10 03:48:05.618588 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612506 2581 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 10 03:48:05.618588 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612508 2581 flags.go:64] FLAG: --image-service-endpoint="" Sep 10 03:48:05.618588 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612512 2581 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 10 03:48:05.618588 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612514 2581 flags.go:64] FLAG: --kube-api-burst="100" Sep 10 03:48:05.618588 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612518 2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 10 03:48:05.619538 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612521 2581 flags.go:64] FLAG: --kube-api-qps="50" Sep 10 03:48:05.619538 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612524 2581 flags.go:64] FLAG: --kube-reserved="" Sep 10 03:48:05.619538 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612526 2581 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 10 03:48:05.619538 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612529 2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 10 03:48:05.619538 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612532 2581 flags.go:64] FLAG: --kubelet-cgroups="" Sep 10 03:48:05.619538 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612535 2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 10 03:48:05.619538 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612539 2581 flags.go:64] FLAG: --lock-file="" Sep 10 03:48:05.619538 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612541 2581 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 10 03:48:05.619538 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612544 2581 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 10 03:48:05.619538 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612547 2581 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 10 03:48:05.619538 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612552 2581 flags.go:64] FLAG: --log-json-split-stream="false" Sep 10 03:48:05.619538 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612555 2581 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 10 03:48:05.619538 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612557 2581 flags.go:64] FLAG: --log-text-split-stream="false" Sep 10 03:48:05.619538 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612560 2581 flags.go:64] FLAG: --logging-format="text" Sep 10 03:48:05.619538 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612562 2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 10 03:48:05.619538 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612566 2581 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 10 03:48:05.619538 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612568 2581 flags.go:64] FLAG: --manifest-url="" Sep 10 03:48:05.619538 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612571 2581 flags.go:64] FLAG: --manifest-url-header="" Sep 10 03:48:05.619538 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612575 2581 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 10 03:48:05.619538 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612578 2581 flags.go:64] FLAG: --max-open-files="1000000" Sep 10 03:48:05.619538 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612582 2581 flags.go:64] FLAG: --max-pods="110" Sep 10 03:48:05.619538 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612585 2581 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 10 03:48:05.619538 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612588 2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 10 03:48:05.619538 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612590 2581 flags.go:64] FLAG: --memory-manager-policy="None" Sep 10 03:48:05.619538 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612593 2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 10 03:48:05.620612 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612596 2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 10 03:48:05.620612 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612599 2581 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 10 03:48:05.620612 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612601 2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 10 03:48:05.620612 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612608 2581 flags.go:64] FLAG: --node-status-max-images="50" Sep 10 03:48:05.620612 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612611 2581 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 10 03:48:05.620612 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612613 2581 flags.go:64] FLAG: --oom-score-adj="-999" Sep 10 03:48:05.620612 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612616 2581 flags.go:64] FLAG: --pod-cidr="" Sep 10 03:48:05.620612 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612620 2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 10 03:48:05.620612 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612625 2581 flags.go:64] FLAG: --pod-manifest-path="" Sep 10 03:48:05.620612 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612628 2581 flags.go:64] FLAG: --pod-max-pids="-1" Sep 10 03:48:05.620612 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612631 2581 flags.go:64] FLAG: --pods-per-core="0" Sep 10 03:48:05.620612 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612633 2581 flags.go:64] FLAG: --port="10250" Sep 10 03:48:05.620612 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612636 2581 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 10 03:48:05.620612 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612640 2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-04964b118b2d4a561" Sep 10 03:48:05.620612 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612643 2581 flags.go:64] FLAG: --qos-reserved="" Sep 10 03:48:05.620612 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612646 2581 flags.go:64] FLAG: --read-only-port="10255" Sep 10 03:48:05.620612 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612648 2581 flags.go:64] FLAG: --register-node="true" Sep 10 03:48:05.620612 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612651 2581 flags.go:64] FLAG: --register-with-taints="" Sep 10 03:48:05.620612 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612655 2581 flags.go:64] FLAG: --registry-burst="10" Sep 10 03:48:05.620612 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612657 2581 flags.go:64] FLAG: --registry-qps="5" Sep 10 03:48:05.620612 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612660 2581 flags.go:64] FLAG: --reserved-cpus="" Sep 10 03:48:05.620612 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612663 2581 flags.go:64] FLAG: --reserved-memory="" Sep 10 03:48:05.620612 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612667 2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 10 03:48:05.620612 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612670 2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 10 03:48:05.620612 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612672 2581 flags.go:64] FLAG: --rotate-certificates="false" Sep 10 03:48:05.621726 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612675 2581 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 10 03:48:05.621726 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612678 2581 flags.go:64] FLAG: --runonce="false" Sep 10 03:48:05.621726 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612681 2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 10 03:48:05.621726 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612683 2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 10 03:48:05.621726 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612686 2581 flags.go:64] FLAG: --seccomp-default="false" Sep 10 03:48:05.621726 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612689 2581 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 10 03:48:05.621726 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612691 2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 10 03:48:05.621726 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612694 2581 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 10 03:48:05.621726 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612697 2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 10 03:48:05.621726 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612716 2581 flags.go:64] FLAG: --storage-driver-password="root" Sep 10 03:48:05.621726 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612720 2581 flags.go:64] FLAG: --storage-driver-secure="false" Sep 10 03:48:05.621726 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612723 2581 flags.go:64] FLAG: --storage-driver-table="stats" Sep 10 03:48:05.621726 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612726 2581 flags.go:64] FLAG: --storage-driver-user="root" Sep 10 03:48:05.621726 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612729 2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 10 03:48:05.621726 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612732 2581 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 10 03:48:05.621726 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612736 2581 flags.go:64] FLAG: --system-cgroups="" Sep 10 03:48:05.621726 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612738 2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 10 03:48:05.621726 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612745 2581 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 10 03:48:05.621726 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612748 2581 flags.go:64] FLAG: --tls-cert-file="" Sep 10 03:48:05.621726 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612751 2581 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 10 03:48:05.621726 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612756 2581 flags.go:64] FLAG: --tls-min-version="" Sep 10 03:48:05.621726 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612759 2581 flags.go:64] FLAG: --tls-private-key-file="" Sep 10 03:48:05.621726 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612761 2581 flags.go:64] FLAG: --topology-manager-policy="none" Sep 10 03:48:05.621726 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612764 2581 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 10 03:48:05.621726 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612767 2581 flags.go:64] FLAG: --topology-manager-scope="container" Sep 10 03:48:05.622649 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612770 2581 flags.go:64] FLAG: --v="2" Sep 10 03:48:05.622649 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612774 2581 flags.go:64] FLAG: --version="false" Sep 10 03:48:05.622649 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612778 2581 flags.go:64] FLAG: --vmodule="" Sep 10 03:48:05.622649 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612781 2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 10 03:48:05.622649 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.612784 2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 10 03:48:05.622649 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612943 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 03:48:05.622649 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612948 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 03:48:05.622649 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612951 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 03:48:05.622649 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612953 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 03:48:05.622649 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612956 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 03:48:05.622649 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612959 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 03:48:05.622649 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612961 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 03:48:05.622649 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612964 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 03:48:05.622649 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612966 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 03:48:05.622649 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612968 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 03:48:05.622649 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612971 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 03:48:05.622649 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612975 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 03:48:05.622649 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612978 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 03:48:05.622649 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612981 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 03:48:05.622649 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612983 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 03:48:05.623194 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612986 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 03:48:05.623194 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612988 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 03:48:05.623194 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612991 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 03:48:05.623194 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612994 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 03:48:05.623194 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612997 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 03:48:05.623194 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.612999 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 03:48:05.623194 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613002 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 03:48:05.623194 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613005 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 03:48:05.623194 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613007 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 03:48:05.623194 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613010 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 03:48:05.623194 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613013 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 03:48:05.623194 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613015 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 03:48:05.623194 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613019 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 03:48:05.623194 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613022 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 03:48:05.623194 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613025 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 03:48:05.623194 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613028 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 03:48:05.623194 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613031 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 03:48:05.623194 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613034 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 03:48:05.623194 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613037 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 03:48:05.623663 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613039 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 03:48:05.623663 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613042 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 03:48:05.623663 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613044 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 03:48:05.623663 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613046 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 03:48:05.623663 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613049 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 03:48:05.623663 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613051 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 03:48:05.623663 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613054 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 03:48:05.623663 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613056 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 03:48:05.623663 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613059 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 03:48:05.623663 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613061 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 03:48:05.623663 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613063 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 03:48:05.623663 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613066 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 03:48:05.623663 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613068 2581 feature_gate.go:328] unrecognized feature gate: Example Sep 10 03:48:05.623663 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613071 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 03:48:05.623663 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613073 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 03:48:05.623663 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613076 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 03:48:05.623663 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613078 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 03:48:05.623663 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613081 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 03:48:05.623663 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613084 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 03:48:05.623663 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613087 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 03:48:05.624181 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613091 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 03:48:05.624181 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613094 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 03:48:05.624181 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613096 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 03:48:05.624181 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613099 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 03:48:05.624181 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613101 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 03:48:05.624181 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613104 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 03:48:05.624181 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613106 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 03:48:05.624181 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613109 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 03:48:05.624181 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613112 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 03:48:05.624181 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613114 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 03:48:05.624181 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613117 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 03:48:05.624181 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613119 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 03:48:05.624181 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613122 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 03:48:05.624181 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613124 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 03:48:05.624181 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613128 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 03:48:05.624181 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613130 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 03:48:05.624181 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613133 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 03:48:05.624181 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613136 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 03:48:05.624181 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613138 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 03:48:05.624181 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613141 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 03:48:05.624667 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613144 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 03:48:05.624667 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613146 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 03:48:05.624667 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613149 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 03:48:05.624667 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613151 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 03:48:05.624667 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613155 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 03:48:05.624667 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613158 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 03:48:05.624667 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613161 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 03:48:05.624667 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613164 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 03:48:05.624667 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613167 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 03:48:05.624667 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613169 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 03:48:05.624667 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613173 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 03:48:05.624667 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613177 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 03:48:05.624667 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613181 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 03:48:05.624667 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613184 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 03:48:05.624667 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613186 2581 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 03:48:05.624667 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613190 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 03:48:05.624667 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613193 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 03:48:05.624667 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613195 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 03:48:05.624667 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613198 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 03:48:05.624667 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613201 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 03:48:05.625161 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613204 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 03:48:05.625161 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613206 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 03:48:05.625161 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613209 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 03:48:05.625161 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613211 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 03:48:05.625161 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.613214 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 03:48:05.625161 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.613219 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 03:48:05.625161 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.622281 2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 10 03:48:05.625161 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.622300 2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 10 03:48:05.625161 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.623777 2581 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 10 03:48:05.625161 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.623885 2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 10 03:48:05.625161 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.623963 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 03:48:05.625161 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.623969 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 03:48:05.625161 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.623972 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 03:48:05.625161 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.623975 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 03:48:05.625507 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.623978 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 03:48:05.625507 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.623981 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 03:48:05.625507 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.623985 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 03:48:05.625507 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.623988 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 03:48:05.625507 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.623991 2581 feature_gate.go:328] unrecognized feature gate: Example Sep 10 03:48:05.625507 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.623993 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 03:48:05.625507 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.623997 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 03:48:05.625507 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624000 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 03:48:05.625507 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624002 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 03:48:05.625507 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624005 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 03:48:05.625507 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624008 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 03:48:05.625507 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624010 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 03:48:05.625507 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624013 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 03:48:05.625507 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624016 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 03:48:05.625507 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624020 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 03:48:05.625507 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624024 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 03:48:05.625507 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624026 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 03:48:05.625507 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624029 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 03:48:05.625507 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624032 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 03:48:05.625507 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624034 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 03:48:05.626002 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624037 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 03:48:05.626002 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624040 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 03:48:05.626002 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624042 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 03:48:05.626002 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624045 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 03:48:05.626002 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624048 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 03:48:05.626002 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624050 2581 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 03:48:05.626002 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624053 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 03:48:05.626002 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624064 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 03:48:05.626002 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624067 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 03:48:05.626002 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624070 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 03:48:05.626002 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624072 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 03:48:05.626002 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624075 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 03:48:05.626002 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624077 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 03:48:05.626002 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624080 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 03:48:05.626002 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624084 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 03:48:05.626002 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624087 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 03:48:05.626002 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624089 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 03:48:05.626002 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624092 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 03:48:05.626002 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624095 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 03:48:05.626460 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624097 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 03:48:05.626460 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624100 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 03:48:05.626460 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624102 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 03:48:05.626460 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624105 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 03:48:05.626460 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624107 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 03:48:05.626460 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624110 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 03:48:05.626460 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624112 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 03:48:05.626460 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624115 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 03:48:05.626460 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624117 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 03:48:05.626460 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624120 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 03:48:05.626460 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624122 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 03:48:05.626460 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624124 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 03:48:05.626460 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624127 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 03:48:05.626460 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624129 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 03:48:05.626460 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624132 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 03:48:05.626460 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624135 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 03:48:05.626460 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624137 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 03:48:05.626460 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624140 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 03:48:05.626460 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624143 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 03:48:05.626460 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624146 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 03:48:05.627060 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624149 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 03:48:05.627060 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624152 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 03:48:05.627060 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624154 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 03:48:05.627060 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624157 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 03:48:05.627060 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624159 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 03:48:05.627060 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624162 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 03:48:05.627060 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624164 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 03:48:05.627060 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624167 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 03:48:05.627060 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624169 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 03:48:05.627060 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624172 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 03:48:05.627060 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624174 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 03:48:05.627060 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624177 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 03:48:05.627060 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624179 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 03:48:05.627060 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624182 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 03:48:05.627060 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624184 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 03:48:05.627060 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624187 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 03:48:05.627060 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624189 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 03:48:05.627060 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624192 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 03:48:05.627060 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624195 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 03:48:05.627060 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624197 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 03:48:05.627541 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624200 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 03:48:05.627541 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624202 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 03:48:05.627541 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624206 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 03:48:05.627541 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624209 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 03:48:05.627541 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624212 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 03:48:05.627541 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624215 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 03:48:05.627541 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624218 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 03:48:05.627541 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624221 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 03:48:05.627541 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624224 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 03:48:05.627541 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624227 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 03:48:05.627541 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624230 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 03:48:05.627541 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624233 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 03:48:05.627541 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624235 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 03:48:05.627541 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624238 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 03:48:05.627541 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624241 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 03:48:05.627541 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624243 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 03:48:05.627941 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.624248 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 03:48:05.627941 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624383 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 03:48:05.627941 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624388 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 03:48:05.627941 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624391 2581 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 03:48:05.627941 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624394 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 03:48:05.627941 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624397 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 03:48:05.627941 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624399 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 03:48:05.627941 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624402 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 03:48:05.627941 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624405 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 03:48:05.627941 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624408 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 03:48:05.627941 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624411 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 03:48:05.627941 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624413 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 03:48:05.627941 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624416 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 03:48:05.627941 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624419 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 03:48:05.627941 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624422 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 03:48:05.628351 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624424 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 03:48:05.628351 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624427 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 03:48:05.628351 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624431 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 03:48:05.628351 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624434 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 03:48:05.628351 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624438 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 03:48:05.628351 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624441 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 03:48:05.628351 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624443 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 03:48:05.628351 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624447 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 03:48:05.628351 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624451 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 03:48:05.628351 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624453 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 03:48:05.628351 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624457 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 03:48:05.628351 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624460 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 03:48:05.628351 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624462 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 03:48:05.628351 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624465 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 03:48:05.628351 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624467 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 03:48:05.628351 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624470 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 03:48:05.628351 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624472 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 03:48:05.628351 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624475 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 03:48:05.628351 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624478 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 03:48:05.628819 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624481 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 03:48:05.628819 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624483 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 03:48:05.628819 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624486 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 03:48:05.628819 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624488 2581 feature_gate.go:328] unrecognized feature gate: Example Sep 10 03:48:05.628819 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624490 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 03:48:05.628819 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624493 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 03:48:05.628819 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624497 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 03:48:05.628819 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624500 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 03:48:05.628819 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624502 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 03:48:05.628819 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624505 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 03:48:05.628819 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624507 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 03:48:05.628819 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624509 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 03:48:05.628819 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624513 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 03:48:05.628819 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624516 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 03:48:05.628819 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624518 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 03:48:05.628819 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624520 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 03:48:05.628819 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624523 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 03:48:05.628819 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624525 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 03:48:05.628819 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624528 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 03:48:05.628819 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624530 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 03:48:05.629308 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624533 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 03:48:05.629308 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624536 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 03:48:05.629308 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624538 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 03:48:05.629308 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624541 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 03:48:05.629308 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624544 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 03:48:05.629308 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624546 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 03:48:05.629308 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624548 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 03:48:05.629308 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624551 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 03:48:05.629308 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624554 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 03:48:05.629308 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624556 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 03:48:05.629308 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624559 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 03:48:05.629308 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624561 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 03:48:05.629308 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624563 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 03:48:05.629308 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624566 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 03:48:05.629308 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624568 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 03:48:05.629308 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624571 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 03:48:05.629308 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624573 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 03:48:05.629308 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624576 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 03:48:05.629308 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624578 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 03:48:05.629308 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624581 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 03:48:05.629829 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624583 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 03:48:05.629829 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624585 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 03:48:05.629829 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624588 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 03:48:05.629829 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624590 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 03:48:05.629829 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624593 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 03:48:05.629829 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624596 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 03:48:05.629829 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624598 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 03:48:05.629829 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624601 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 03:48:05.629829 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624603 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 03:48:05.629829 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624605 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 03:48:05.629829 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624608 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 03:48:05.629829 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624610 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 03:48:05.629829 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624613 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 03:48:05.629829 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624615 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 03:48:05.629829 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624617 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 03:48:05.629829 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624620 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 03:48:05.629829 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624623 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 03:48:05.629829 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624625 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 03:48:05.629829 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624628 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 03:48:05.629829 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624630 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 03:48:05.630305 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624633 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 03:48:05.630305 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624635 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 03:48:05.630305 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624638 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 03:48:05.630305 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624640 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 03:48:05.630305 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624642 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 03:48:05.630305 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:05.624645 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 03:48:05.630305 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.624650 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 03:48:05.630305 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.624778 2581 server.go:962] "Client rotation is on, will bootstrap in background" Sep 10 03:48:05.630305 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.627987 2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 10 03:48:05.630305 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.628950 2581 server.go:1019] "Starting client certificate rotation" Sep 10 03:48:05.630305 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.629054 2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 03:48:05.630305 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.629084 2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 03:48:05.632013 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.632000 2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 10 03:48:05.633148 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.633129 2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 10 03:48:05.642383 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.642364 2581 log.go:25] "Validated CRI v1 runtime API" Sep 10 03:48:05.645371 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.645357 2581 log.go:25] "Validated CRI v1 image API" Sep 10 03:48:05.647900 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.647886 2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 10 03:48:05.650900 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.650882 2581 fs.go:135] Filesystem UUIDs: map[0271d213-c062-404b-8ea9-9c6678fdf8bc:/dev/nvme0n1p3 5a67079b-1cb1-4318-bd48-a0df775f480d:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Sep 10 03:48:05.650960 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.650901 2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 10 03:48:05.657271 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.657035 2581 manager.go:216] Machine: {Timestamp:2026-09-10 03:48:05.655649312 +0000 UTC m=+0.314294500 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3105061 MemoryCapacity:32812154880 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec21d40be75e9776cf0a33896a10bfa9 SystemUUID:ec21d40b-e75e-9776-cf0a-33896a10bfa9 BootID:c8661f64-6966-4d14-bc61-c59d5811c289 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406077440 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:72:37:d9:1e:2b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:72:37:d9:1e:2b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:02:09:5c:3d:ac:53 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812154880 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 10 03:48:05.657350 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.657272 2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 10 03:48:05.657384 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.657349 2581 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 10 03:48:05.659952 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.659922 2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 10 03:48:05.660098 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.659952 2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-77.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 10 03:48:05.660680 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.660670 2581 topology_manager.go:138] "Creating topology manager with none policy" Sep 10 03:48:05.660730 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.660682 2581 container_manager_linux.go:309] "Creating device plugin manager" Sep 10 03:48:05.660730 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.660690 2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 10 03:48:05.661273 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.661258 2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 10 03:48:05.661309 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.661299 2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 10 03:48:05.662073 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.662062 2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 10 03:48:05.665897 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.665886 2581 state_mem.go:36] "Initialized new in-memory state store" Sep 10 03:48:05.666022 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.666013 2581 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 10 03:48:05.666065 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.666048 2581 kubelet.go:486] "Attempting to sync node with API server" Sep 10 03:48:05.666094 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.666065 2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 10 03:48:05.666094 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.666079 2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 10 03:48:05.666094 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.666088 2581 kubelet.go:398] "Adding apiserver pod source" Sep 10 03:48:05.666218 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.666096 2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 10 03:48:05.667306 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.667288 2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 10 03:48:05.667398 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.667316 2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 10 03:48:05.670024 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.670001 2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 10 03:48:05.672749 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.672731 2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 10 03:48:05.672796 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.672766 2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 10 03:48:05.673826 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.673815 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 10 03:48:05.673879 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.673832 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 10 03:48:05.673879 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.673838 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 10 03:48:05.673879 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.673843 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 10 03:48:05.673879 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.673852 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 10 03:48:05.673879 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.673858 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 10 03:48:05.673879 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.673872 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 10 03:48:05.673879 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.673877 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 10 03:48:05.674050 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.673885 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 10 03:48:05.674050 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.673891 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 10 03:48:05.674050 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.673900 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 10 03:48:05.674050 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.673905 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 10 03:48:05.674612 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.674604 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 10 03:48:05.674612 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.674613 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 10 03:48:05.677818 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.677788 2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-77.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 03:48:05.677899 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:05.677814 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-77.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 10 03:48:05.678016 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:05.677907 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 10 03:48:05.678424 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.678410 2581 server.go:1268] "Started kubelet" Sep 10 03:48:05.678526 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.678483 2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 10 03:48:05.678576 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.678547 2581 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 10 03:48:05.678620 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.678501 2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 10 03:48:05.679405 ip-10-0-142-77 systemd[1]: Started Kubernetes Kubelet. Sep 10 03:48:05.679676 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.679570 2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 10 03:48:05.681158 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.681144 2581 server.go:310] "Adding debug handlers to kubelet server" Sep 10 03:48:05.683087 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:05.683068 2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 10 03:48:05.683792 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:05.682884 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-77.ec2.internal.18d3d90369bd9f00 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-77.ec2.internal,UID:ip-10-0-142-77.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-77.ec2.internal,},FirstTimestamp:2026-09-10 03:48:05.678382848 +0000 UTC m=+0.337028034,LastTimestamp:2026-09-10 03:48:05.678382848 +0000 UTC m=+0.337028034,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-77.ec2.internal,}" Sep 10 03:48:05.685816 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.685796 2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 10 03:48:05.685816 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.685806 2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 10 03:48:05.686029 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:05.686013 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-77.ec2.internal\" not found" Sep 10 03:48:05.686478 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.686451 2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 10 03:48:05.686478 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.686453 2581 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 10 03:48:05.686618 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.686488 2581 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 10 03:48:05.686676 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.686664 2581 reconstruct.go:97] "Volume reconstruction finished" Sep 10 03:48:05.686754 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.686679 2581 reconciler.go:29] "Reconciler: start to sync state" Sep 10 03:48:05.686841 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.686825 2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 10 03:48:05.686909 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.686845 2581 factory.go:55] Registering systemd factory Sep 10 03:48:05.686909 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.686855 2581 factory.go:223] Registration of the systemd container factory successfully Sep 10 03:48:05.687042 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.687029 2581 factory.go:153] Registering CRI-O factory Sep 10 03:48:05.687085 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.687047 2581 factory.go:223] Registration of the crio container factory successfully Sep 10 03:48:05.687085 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.687064 2581 factory.go:103] Registering Raw factory Sep 10 03:48:05.687085 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.687078 2581 manager.go:1196] Started watching for new ooms in manager Sep 10 03:48:05.687518 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.687505 2581 manager.go:349] Starting recovery of all containers Sep 10 03:48:05.690368 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:05.690342 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 10 03:48:05.690494 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:05.690416 2581 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-77.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 10 03:48:05.691484 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.691456 2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9hgvd" Sep 10 03:48:05.699941 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.699925 2581 manager.go:354] Recovery completed Sep 10 03:48:05.700532 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.700510 2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9hgvd" Sep 10 03:48:05.704129 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.704116 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 03:48:05.706295 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.706278 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-77.ec2.internal" event="NodeHasSufficientMemory" Sep 10 03:48:05.706365 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.706312 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-77.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 03:48:05.706365 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.706326 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-77.ec2.internal" event="NodeHasSufficientPID" Sep 10 03:48:05.706891 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.706878 2581 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 10 03:48:05.706938 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.706890 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 03:48:05.706938 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.706892 2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 10 03:48:05.706938 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.706937 2581 state_mem.go:36] "Initialized new in-memory state store" Sep 10 03:48:05.708069 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.708053 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-77.ec2.internal" event="NodeHasSufficientMemory" Sep 10 03:48:05.708116 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.708080 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-77.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 03:48:05.708116 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.708091 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-77.ec2.internal" event="NodeHasSufficientPID" Sep 10 03:48:05.708531 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:05.708461 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-77.ec2.internal.18d3d9036b67898e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-77.ec2.internal,UID:ip-10-0-142-77.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-77.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-77.ec2.internal,},FirstTimestamp:2026-09-10 03:48:05.706295694 +0000 UTC m=+0.364940886,LastTimestamp:2026-09-10 03:48:05.706295694 +0000 UTC m=+0.364940886,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-77.ec2.internal,}" Sep 10 03:48:05.708961 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.708948 2581 policy_none.go:49] "None policy: Start" Sep 10 03:48:05.709022 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.708967 2581 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 10 03:48:05.709022 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.708980 2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 10 03:48:05.711019 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.711005 2581 policy_none.go:47] "Start" Sep 10 03:48:05.750464 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.750402 2581 manager.go:337] "Starting Device Plugin manager" Sep 10 03:48:05.750464 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:05.750444 2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 10 03:48:05.750464 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.750457 2581 server.go:87] "Starting device plugin registration server" Sep 10 03:48:05.776881 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.750722 2581 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 10 03:48:05.776881 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.750741 2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 10 03:48:05.776881 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.750892 2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 10 03:48:05.776881 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.750990 2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 10 03:48:05.776881 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.751017 2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 10 03:48:05.776881 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:05.751436 2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 10 03:48:05.776881 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:05.751476 2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-77.ec2.internal\" not found" Sep 10 03:48:05.801546 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.801521 2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 10 03:48:05.802811 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.802793 2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 10 03:48:05.802908 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.802843 2581 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 10 03:48:05.802908 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.802867 2581 kubelet.go:2448] "Starting kubelet main sync loop" Sep 10 03:48:05.803001 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:05.802909 2581 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 10 03:48:05.804904 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.804881 2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 03:48:05.851237 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.851218 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 03:48:05.852076 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.852060 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-77.ec2.internal" event="NodeHasSufficientMemory" Sep 10 03:48:05.852175 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.852097 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-77.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 03:48:05.852175 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.852113 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-77.ec2.internal" event="NodeHasSufficientPID" Sep 10 03:48:05.852175 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.852149 2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-77.ec2.internal" Sep 10 03:48:05.862538 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.862453 2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-77.ec2.internal" Sep 10 03:48:05.862538 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:05.862475 2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-77.ec2.internal\": node \"ip-10-0-142-77.ec2.internal\" not found" Sep 10 03:48:05.875319 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:05.875295 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-77.ec2.internal\" not found" Sep 10 03:48:05.903174 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.903144 2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-77.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-77.ec2.internal"] Sep 10 03:48:05.903268 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.903217 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 03:48:05.904591 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.904577 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-77.ec2.internal" event="NodeHasSufficientMemory" Sep 10 03:48:05.904654 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.904604 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-77.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 03:48:05.904654 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.904614 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-77.ec2.internal" event="NodeHasSufficientPID" Sep 10 03:48:05.906846 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.906835 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 03:48:05.906960 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.906947 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-77.ec2.internal" Sep 10 03:48:05.907001 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.906983 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 03:48:05.907496 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.907481 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-77.ec2.internal" event="NodeHasSufficientMemory" Sep 10 03:48:05.907571 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.907509 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-77.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 03:48:05.907571 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.907524 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-77.ec2.internal" event="NodeHasSufficientPID" Sep 10 03:48:05.907663 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.907593 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-77.ec2.internal" event="NodeHasSufficientMemory" Sep 10 03:48:05.907663 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.907612 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-77.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 03:48:05.907663 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.907622 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-77.ec2.internal" event="NodeHasSufficientPID" Sep 10 03:48:05.909814 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.909797 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-77.ec2.internal" Sep 10 03:48:05.909897 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.909831 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 03:48:05.910467 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.910451 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-77.ec2.internal" event="NodeHasSufficientMemory" Sep 10 03:48:05.910546 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.910484 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-77.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 03:48:05.910546 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.910499 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-77.ec2.internal" event="NodeHasSufficientPID" Sep 10 03:48:05.937351 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:05.937334 2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-77.ec2.internal\" not found" node="ip-10-0-142-77.ec2.internal" Sep 10 03:48:05.941893 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:05.941880 2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-77.ec2.internal\" not found" node="ip-10-0-142-77.ec2.internal" Sep 10 03:48:05.975916 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:05.975893 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-77.ec2.internal\" not found" Sep 10 03:48:05.988290 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.988264 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6524cf98e47d7fba2fecd528cfc30ded-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-77.ec2.internal\" (UID: \"6524cf98e47d7fba2fecd528cfc30ded\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-77.ec2.internal" Sep 10 03:48:05.988388 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.988296 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/81cf177ab4229900eb060a4c89e0a554-config\") pod \"kube-apiserver-proxy-ip-10-0-142-77.ec2.internal\" (UID: \"81cf177ab4229900eb060a4c89e0a554\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-77.ec2.internal" Sep 10 03:48:05.988388 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:05.988320 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6524cf98e47d7fba2fecd528cfc30ded-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-77.ec2.internal\" (UID: \"6524cf98e47d7fba2fecd528cfc30ded\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-77.ec2.internal" Sep 10 03:48:06.076514 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:06.076478 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-77.ec2.internal\" not found" Sep 10 03:48:06.088850 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:06.088827 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6524cf98e47d7fba2fecd528cfc30ded-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-77.ec2.internal\" (UID: \"6524cf98e47d7fba2fecd528cfc30ded\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-77.ec2.internal" Sep 10 03:48:06.088928 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:06.088879 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6524cf98e47d7fba2fecd528cfc30ded-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-77.ec2.internal\" (UID: \"6524cf98e47d7fba2fecd528cfc30ded\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-77.ec2.internal" Sep 10 03:48:06.088928 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:06.088904 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6524cf98e47d7fba2fecd528cfc30ded-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-77.ec2.internal\" (UID: \"6524cf98e47d7fba2fecd528cfc30ded\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-77.ec2.internal" Sep 10 03:48:06.088928 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:06.088920 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6524cf98e47d7fba2fecd528cfc30ded-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-77.ec2.internal\" (UID: \"6524cf98e47d7fba2fecd528cfc30ded\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-77.ec2.internal" Sep 10 03:48:06.089030 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:06.088937 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/81cf177ab4229900eb060a4c89e0a554-config\") pod \"kube-apiserver-proxy-ip-10-0-142-77.ec2.internal\" (UID: \"81cf177ab4229900eb060a4c89e0a554\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-77.ec2.internal" Sep 10 03:48:06.089030 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:06.088967 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/81cf177ab4229900eb060a4c89e0a554-config\") pod \"kube-apiserver-proxy-ip-10-0-142-77.ec2.internal\" (UID: \"81cf177ab4229900eb060a4c89e0a554\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-77.ec2.internal" Sep 10 03:48:06.177267 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:06.177239 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-77.ec2.internal\" not found" Sep 10 03:48:06.239791 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:06.239761 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-77.ec2.internal" Sep 10 03:48:06.245436 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:06.245418 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-77.ec2.internal" Sep 10 03:48:06.277778 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:06.277752 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-77.ec2.internal\" not found" Sep 10 03:48:06.378324 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:06.378279 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-77.ec2.internal\" not found" Sep 10 03:48:06.478890 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:06.478829 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-77.ec2.internal\" not found" Sep 10 03:48:06.579312 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:06.579286 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-77.ec2.internal\" not found" Sep 10 03:48:06.628857 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:06.628829 2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 10 03:48:06.629254 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:06.628979 2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 10 03:48:06.679740 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:06.679718 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-77.ec2.internal\" not found" Sep 10 03:48:06.705536 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:06.705503 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 03:43:06 +0000 UTC" deadline="2028-05-29 11:10:11.297618272 +0000 UTC" Sep 10 03:48:06.705536 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:06.705532 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15055h22m4.592089428s" Sep 10 03:48:06.749868 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:06.749809 2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 03:48:06.780661 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:06.780635 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-77.ec2.internal\" not found" Sep 10 03:48:06.805221 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:06.805188 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81cf177ab4229900eb060a4c89e0a554.slice/crio-79501ce27c9dc7227dc933521d3c38fbdd5d37326e4751e9c137abc17dbac1a3 WatchSource:0}: Error finding container 79501ce27c9dc7227dc933521d3c38fbdd5d37326e4751e9c137abc17dbac1a3: Status 404 returned error can't find the container with id 79501ce27c9dc7227dc933521d3c38fbdd5d37326e4751e9c137abc17dbac1a3 Sep 10 03:48:06.805520 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:06.805494 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6524cf98e47d7fba2fecd528cfc30ded.slice/crio-975831ef3f37d69b3fac7170df9851c711dc66b97e7bed2f6cc89a8a27a73f4e WatchSource:0}: Error finding container 975831ef3f37d69b3fac7170df9851c711dc66b97e7bed2f6cc89a8a27a73f4e: Status 404 returned error can't find the container with id 975831ef3f37d69b3fac7170df9851c711dc66b97e7bed2f6cc89a8a27a73f4e Sep 10 03:48:06.808978 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:06.808963 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 03:48:06.881195 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:06.881168 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-77.ec2.internal\" not found" Sep 10 03:48:06.908414 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:06.908392 2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 03:48:06.958500 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:06.958477 2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 03:48:06.986408 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:06.986387 2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-77.ec2.internal" Sep 10 03:48:06.994165 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:06.994151 2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 10 03:48:06.995032 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:06.995021 2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-77.ec2.internal" Sep 10 03:48:07.004666 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:07.004627 2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 10 03:48:08.800496 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.800467 2581 apiserver.go:52] "Watching apiserver" Sep 10 03:48:08.808348 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.808319 2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 10 03:48:08.810198 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.810175 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-cqmkb","kube-system/global-pull-secret-syncer-2z94t","kube-system/kube-apiserver-proxy-ip-10-0-142-77.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rrqkg","openshift-cluster-node-tuning-operator/tuned-kkrg5","openshift-dns/node-resolver-7lpdd","openshift-image-registry/node-ca-r58nd","openshift-multus/multus-additional-cni-plugins-6tbmr","openshift-multus/multus-wnlbc","kube-system/konnectivity-agent-sjkng","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-77.ec2.internal","openshift-multus/network-metrics-daemon-htpt9","openshift-network-diagnostics/network-check-target-7r9qf","openshift-network-operator/iptables-alerter-tnrwv"] Sep 10 03:48:08.813134 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.813116 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.815509 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.815067 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-sjkng" Sep 10 03:48:08.815509 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.815368 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 10 03:48:08.815509 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.815493 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 10 03:48:08.815702 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.815595 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 10 03:48:08.815771 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.815753 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-c28mh\"" Sep 10 03:48:08.815811 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.815768 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 10 03:48:08.817196 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.817177 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 10 03:48:08.817282 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.817216 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-jwtgd\"" Sep 10 03:48:08.817446 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.817429 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 10 03:48:08.817577 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.817558 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rrqkg" Sep 10 03:48:08.819581 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.819561 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 10 03:48:08.819674 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.819601 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.820188 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.820170 2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 10 03:48:08.820271 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.820233 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 10 03:48:08.820329 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.820289 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 10 03:48:08.826500 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.825327 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:08.826500 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.825799 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-5qcs5\"" Sep 10 03:48:08.826500 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.826121 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 10 03:48:08.826500 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.826171 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 10 03:48:08.826500 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.826338 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 10 03:48:08.826754 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.826510 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 10 03:48:08.826754 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.826689 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 10 03:48:08.826754 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.826744 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 10 03:48:08.827331 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.826953 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7lpdd" Sep 10 03:48:08.827331 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.827073 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-pjs29\"" Sep 10 03:48:08.827663 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.827643 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 10 03:48:08.827818 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.827791 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 10 03:48:08.828201 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.828180 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-72khx\"" Sep 10 03:48:08.828473 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.828439 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-r58nd" Sep 10 03:48:08.830091 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.830071 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-host-cni-netd\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.830198 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.830104 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tfhr\" (UniqueName: \"kubernetes.io/projected/b4c0f4fb-e2d6-4ed5-969e-ec1ff4aaff92-kube-api-access-7tfhr\") pod \"node-ca-r58nd\" (UID: \"b4c0f4fb-e2d6-4ed5-969e-ec1ff4aaff92\") " pod="openshift-image-registry/node-ca-r58nd" Sep 10 03:48:08.830198 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.830130 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f9d9771e-7f4b-4fd5-ac4a-f44b44379f17-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rrqkg\" (UID: \"f9d9771e-7f4b-4fd5-ac4a-f44b44379f17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rrqkg" Sep 10 03:48:08.830198 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.830155 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1d9fdaf3-79da-4413-8a44-346fe29074be-multus-socket-dir-parent\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.830198 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.830189 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1d9fdaf3-79da-4413-8a44-346fe29074be-host-run-multus-certs\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.830469 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.830214 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 10 03:48:08.830469 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.830214 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-log-socket\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.830469 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.830342 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/289045b6-dd39-4301-ac33-3f4fe69ad152-lib-modules\") pod \"tuned-kkrg5\" (UID: \"289045b6-dd39-4301-ac33-3f4fe69ad152\") " pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:08.830469 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.830365 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29111c2f-bef9-4076-9014-7f041915d6df-tmp-dir\") pod \"node-resolver-7lpdd\" (UID: \"29111c2f-bef9-4076-9014-7f041915d6df\") " pod="openshift-dns/node-resolver-7lpdd" Sep 10 03:48:08.830469 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.830386 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/289045b6-dd39-4301-ac33-3f4fe69ad152-etc-sysconfig\") pod \"tuned-kkrg5\" (UID: \"289045b6-dd39-4301-ac33-3f4fe69ad152\") " pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:08.830469 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.830407 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/289045b6-dd39-4301-ac33-3f4fe69ad152-etc-kubernetes\") pod \"tuned-kkrg5\" (UID: \"289045b6-dd39-4301-ac33-3f4fe69ad152\") " pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:08.830469 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.830427 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/289045b6-dd39-4301-ac33-3f4fe69ad152-var-lib-kubelet\") pod \"tuned-kkrg5\" (UID: \"289045b6-dd39-4301-ac33-3f4fe69ad152\") " pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:08.830469 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.830450 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-systemd-units\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.830921 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.830472 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-host-run-netns\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.830921 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.830494 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f0b85b12-5116-477e-86d0-2946bc920ded-ovnkube-script-lib\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.830921 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.830535 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/289045b6-dd39-4301-ac33-3f4fe69ad152-etc-modprobe-d\") pod \"tuned-kkrg5\" (UID: \"289045b6-dd39-4301-ac33-3f4fe69ad152\") " pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:08.830921 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.830589 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 10 03:48:08.830921 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.830588 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/289045b6-dd39-4301-ac33-3f4fe69ad152-etc-tuned\") pod \"tuned-kkrg5\" (UID: \"289045b6-dd39-4301-ac33-3f4fe69ad152\") " pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:08.830921 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.830662 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1d9fdaf3-79da-4413-8a44-346fe29074be-multus-cni-dir\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.830921 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.830686 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1d9fdaf3-79da-4413-8a44-346fe29074be-cni-binary-copy\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.830921 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.830706 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1d9fdaf3-79da-4413-8a44-346fe29074be-multus-conf-dir\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.830921 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.830728 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f0b85b12-5116-477e-86d0-2946bc920ded-env-overrides\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.830921 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.830765 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/29111c2f-bef9-4076-9014-7f041915d6df-hosts-file\") pod \"node-resolver-7lpdd\" (UID: \"29111c2f-bef9-4076-9014-7f041915d6df\") " pod="openshift-dns/node-resolver-7lpdd" Sep 10 03:48:08.830921 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.830789 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/289045b6-dd39-4301-ac33-3f4fe69ad152-etc-sysctl-conf\") pod \"tuned-kkrg5\" (UID: \"289045b6-dd39-4301-ac33-3f4fe69ad152\") " pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:08.830921 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.830810 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/289045b6-dd39-4301-ac33-3f4fe69ad152-etc-systemd\") pod \"tuned-kkrg5\" (UID: \"289045b6-dd39-4301-ac33-3f4fe69ad152\") " pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:08.830921 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.830831 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f9d9771e-7f4b-4fd5-ac4a-f44b44379f17-etc-selinux\") pod \"aws-ebs-csi-driver-node-rrqkg\" (UID: \"f9d9771e-7f4b-4fd5-ac4a-f44b44379f17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rrqkg" Sep 10 03:48:08.830921 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.830872 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f9d9771e-7f4b-4fd5-ac4a-f44b44379f17-sys-fs\") pod \"aws-ebs-csi-driver-node-rrqkg\" (UID: \"f9d9771e-7f4b-4fd5-ac4a-f44b44379f17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rrqkg" Sep 10 03:48:08.830921 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.830905 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1d9fdaf3-79da-4413-8a44-346fe29074be-etc-kubernetes\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.830921 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.830928 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-host-slash\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.831807 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.830950 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-host-run-ovn-kubernetes\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.831807 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.830992 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-var-lib-openvswitch\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.831807 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.831022 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.831807 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.831045 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f0b85b12-5116-477e-86d0-2946bc920ded-ovnkube-config\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.831807 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.831067 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b4c0f4fb-e2d6-4ed5-969e-ec1ff4aaff92-host\") pod \"node-ca-r58nd\" (UID: \"b4c0f4fb-e2d6-4ed5-969e-ec1ff4aaff92\") " pod="openshift-image-registry/node-ca-r58nd" Sep 10 03:48:08.831807 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.831076 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-2n9gp\"" Sep 10 03:48:08.831807 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.831092 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/271dd21f-0eed-4858-810a-687acbea64ec-agent-certs\") pod \"konnectivity-agent-sjkng\" (UID: \"271dd21f-0eed-4858-810a-687acbea64ec\") " pod="kube-system/konnectivity-agent-sjkng" Sep 10 03:48:08.831807 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.831118 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kt8rf\" (UniqueName: \"kubernetes.io/projected/f9d9771e-7f4b-4fd5-ac4a-f44b44379f17-kube-api-access-kt8rf\") pod \"aws-ebs-csi-driver-node-rrqkg\" (UID: \"f9d9771e-7f4b-4fd5-ac4a-f44b44379f17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rrqkg" Sep 10 03:48:08.831807 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.831135 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 10 03:48:08.831807 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.831140 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-host-kubelet\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.831807 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.831255 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-run-ovn\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.831807 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.831279 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-node-log\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.831807 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.831305 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b4c0f4fb-e2d6-4ed5-969e-ec1ff4aaff92-serviceca\") pod \"node-ca-r58nd\" (UID: \"b4c0f4fb-e2d6-4ed5-969e-ec1ff4aaff92\") " pod="openshift-image-registry/node-ca-r58nd" Sep 10 03:48:08.831807 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.831339 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1d9fdaf3-79da-4413-8a44-346fe29074be-system-cni-dir\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.831807 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.831373 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 10 03:48:08.831807 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.831376 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-nvfb8\"" Sep 10 03:48:08.831807 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.831377 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1d9fdaf3-79da-4413-8a44-346fe29074be-hostroot\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.831807 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.831472 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-run-openvswitch\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.831807 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.831497 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/271dd21f-0eed-4858-810a-687acbea64ec-konnectivity-ca\") pod \"konnectivity-agent-sjkng\" (UID: \"271dd21f-0eed-4858-810a-687acbea64ec\") " pod="kube-system/konnectivity-agent-sjkng" Sep 10 03:48:08.832546 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.831526 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1d9fdaf3-79da-4413-8a44-346fe29074be-cnibin\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.832546 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.831551 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1d9fdaf3-79da-4413-8a44-346fe29074be-host-var-lib-cni-multus\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.832546 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.831584 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 10 03:48:08.832546 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.831584 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmczd\" (UniqueName: \"kubernetes.io/projected/1d9fdaf3-79da-4413-8a44-346fe29074be-kube-api-access-wmczd\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.832546 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.831648 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5579d\" (UniqueName: \"kubernetes.io/projected/f0b85b12-5116-477e-86d0-2946bc920ded-kube-api-access-5579d\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.832546 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.831675 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-etc-openvswitch\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.832546 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.831704 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/289045b6-dd39-4301-ac33-3f4fe69ad152-run\") pod \"tuned-kkrg5\" (UID: \"289045b6-dd39-4301-ac33-3f4fe69ad152\") " pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:08.832546 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.831732 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1d9fdaf3-79da-4413-8a44-346fe29074be-multus-daemon-config\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.832546 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.831755 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f9d9771e-7f4b-4fd5-ac4a-f44b44379f17-registration-dir\") pod \"aws-ebs-csi-driver-node-rrqkg\" (UID: \"f9d9771e-7f4b-4fd5-ac4a-f44b44379f17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rrqkg" Sep 10 03:48:08.832546 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.831779 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/289045b6-dd39-4301-ac33-3f4fe69ad152-sys\") pod \"tuned-kkrg5\" (UID: \"289045b6-dd39-4301-ac33-3f4fe69ad152\") " pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:08.832546 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.831802 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/289045b6-dd39-4301-ac33-3f4fe69ad152-host\") pod \"tuned-kkrg5\" (UID: \"289045b6-dd39-4301-ac33-3f4fe69ad152\") " pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:08.832546 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.831837 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfxkj\" (UniqueName: \"kubernetes.io/projected/29111c2f-bef9-4076-9014-7f041915d6df-kube-api-access-bfxkj\") pod \"node-resolver-7lpdd\" (UID: \"29111c2f-bef9-4076-9014-7f041915d6df\") " pod="openshift-dns/node-resolver-7lpdd" Sep 10 03:48:08.832546 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.831859 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f9d9771e-7f4b-4fd5-ac4a-f44b44379f17-device-dir\") pod \"aws-ebs-csi-driver-node-rrqkg\" (UID: \"f9d9771e-7f4b-4fd5-ac4a-f44b44379f17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rrqkg" Sep 10 03:48:08.832546 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.831881 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1d9fdaf3-79da-4413-8a44-346fe29074be-host-var-lib-cni-bin\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.832546 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.831902 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/289045b6-dd39-4301-ac33-3f4fe69ad152-etc-sysctl-d\") pod \"tuned-kkrg5\" (UID: \"289045b6-dd39-4301-ac33-3f4fe69ad152\") " pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:08.832546 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.831923 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/289045b6-dd39-4301-ac33-3f4fe69ad152-tmp\") pod \"tuned-kkrg5\" (UID: \"289045b6-dd39-4301-ac33-3f4fe69ad152\") " pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:08.832546 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.831963 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1d9fdaf3-79da-4413-8a44-346fe29074be-host-run-netns\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.833209 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.832026 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1d9fdaf3-79da-4413-8a44-346fe29074be-host-run-k8s-cni-cncf-io\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.833209 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.832066 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f9d9771e-7f4b-4fd5-ac4a-f44b44379f17-socket-dir\") pod \"aws-ebs-csi-driver-node-rrqkg\" (UID: \"f9d9771e-7f4b-4fd5-ac4a-f44b44379f17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rrqkg" Sep 10 03:48:08.833209 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.832093 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-run-systemd\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.833209 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.832118 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f0b85b12-5116-477e-86d0-2946bc920ded-ovn-node-metrics-cert\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.833209 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.832143 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2ld7\" (UniqueName: \"kubernetes.io/projected/289045b6-dd39-4301-ac33-3f4fe69ad152-kube-api-access-p2ld7\") pod \"tuned-kkrg5\" (UID: \"289045b6-dd39-4301-ac33-3f4fe69ad152\") " pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:08.833209 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.832168 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1d9fdaf3-79da-4413-8a44-346fe29074be-host-var-lib-kubelet\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.833209 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.832189 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-host-cni-bin\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.833209 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.832672 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6tbmr" Sep 10 03:48:08.834850 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.834834 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 10 03:48:08.834956 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.834865 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-db8gj\"" Sep 10 03:48:08.835148 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.835131 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z94t" Sep 10 03:48:08.835278 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:08.835217 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z94t" podUID="72f457e8-c3eb-4b62-81e6-10cdd0f78d63" Sep 10 03:48:08.836467 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.835404 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 10 03:48:08.836544 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.836532 2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 10 03:48:08.837277 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.837261 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htpt9" Sep 10 03:48:08.837378 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:08.837337 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htpt9" podUID="fca9ba6d-4934-47e4-8be5-4296c7a6c530" Sep 10 03:48:08.839620 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.839605 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7r9qf" Sep 10 03:48:08.839708 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:08.839659 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7r9qf" podUID="6be9cc5c-8c3c-4985-98ab-ec0848534ff0" Sep 10 03:48:08.842762 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.842742 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tnrwv" Sep 10 03:48:08.845304 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.845146 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 10 03:48:08.845304 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.845194 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 10 03:48:08.845304 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.845203 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 10 03:48:08.845542 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.845518 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-dwkww\"" Sep 10 03:48:08.857385 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.857368 2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-48plt" Sep 10 03:48:08.865259 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.865242 2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-48plt" Sep 10 03:48:08.920693 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.920656 2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 10 03:48:08.933202 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.933161 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b4c0f4fb-e2d6-4ed5-969e-ec1ff4aaff92-host\") pod \"node-ca-r58nd\" (UID: \"b4c0f4fb-e2d6-4ed5-969e-ec1ff4aaff92\") " pod="openshift-image-registry/node-ca-r58nd" Sep 10 03:48:08.933362 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.933206 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/24e969cd-fc73-4628-bcd8-0abb930691d8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6tbmr\" (UID: \"24e969cd-fc73-4628-bcd8-0abb930691d8\") " pod="openshift-multus/multus-additional-cni-plugins-6tbmr" Sep 10 03:48:08.933362 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.933233 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/72f457e8-c3eb-4b62-81e6-10cdd0f78d63-original-pull-secret\") pod \"global-pull-secret-syncer-2z94t\" (UID: \"72f457e8-c3eb-4b62-81e6-10cdd0f78d63\") " pod="kube-system/global-pull-secret-syncer-2z94t" Sep 10 03:48:08.933362 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.933251 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjq6z\" (UniqueName: \"kubernetes.io/projected/fca9ba6d-4934-47e4-8be5-4296c7a6c530-kube-api-access-fjq6z\") pod \"network-metrics-daemon-htpt9\" (UID: \"fca9ba6d-4934-47e4-8be5-4296c7a6c530\") " pod="openshift-multus/network-metrics-daemon-htpt9" Sep 10 03:48:08.933362 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.933270 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/271dd21f-0eed-4858-810a-687acbea64ec-agent-certs\") pod \"konnectivity-agent-sjkng\" (UID: \"271dd21f-0eed-4858-810a-687acbea64ec\") " pod="kube-system/konnectivity-agent-sjkng" Sep 10 03:48:08.933362 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.933286 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kt8rf\" (UniqueName: \"kubernetes.io/projected/f9d9771e-7f4b-4fd5-ac4a-f44b44379f17-kube-api-access-kt8rf\") pod \"aws-ebs-csi-driver-node-rrqkg\" (UID: \"f9d9771e-7f4b-4fd5-ac4a-f44b44379f17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rrqkg" Sep 10 03:48:08.933362 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.933300 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-host-kubelet\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.933362 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.933315 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-run-ovn\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.933362 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.933338 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-node-log\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.933362 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.933358 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b4c0f4fb-e2d6-4ed5-969e-ec1ff4aaff92-serviceca\") pod \"node-ca-r58nd\" (UID: \"b4c0f4fb-e2d6-4ed5-969e-ec1ff4aaff92\") " pod="openshift-image-registry/node-ca-r58nd" Sep 10 03:48:08.933794 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.933377 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1d9fdaf3-79da-4413-8a44-346fe29074be-system-cni-dir\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.933794 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.933400 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1d9fdaf3-79da-4413-8a44-346fe29074be-hostroot\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.933794 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.933421 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-run-openvswitch\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.933794 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.933443 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdjfj\" (UniqueName: \"kubernetes.io/projected/24e969cd-fc73-4628-bcd8-0abb930691d8-kube-api-access-hdjfj\") pod \"multus-additional-cni-plugins-6tbmr\" (UID: \"24e969cd-fc73-4628-bcd8-0abb930691d8\") " pod="openshift-multus/multus-additional-cni-plugins-6tbmr" Sep 10 03:48:08.933794 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.933458 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/72f457e8-c3eb-4b62-81e6-10cdd0f78d63-kubelet-config\") pod \"global-pull-secret-syncer-2z94t\" (UID: \"72f457e8-c3eb-4b62-81e6-10cdd0f78d63\") " pod="kube-system/global-pull-secret-syncer-2z94t" Sep 10 03:48:08.933794 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.933473 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hc9lh\" (UniqueName: \"kubernetes.io/projected/6be9cc5c-8c3c-4985-98ab-ec0848534ff0-kube-api-access-hc9lh\") pod \"network-check-target-7r9qf\" (UID: \"6be9cc5c-8c3c-4985-98ab-ec0848534ff0\") " pod="openshift-network-diagnostics/network-check-target-7r9qf" Sep 10 03:48:08.933794 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.933488 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/336788f7-971b-4071-80e6-afaa13696a1d-iptables-alerter-script\") pod \"iptables-alerter-tnrwv\" (UID: \"336788f7-971b-4071-80e6-afaa13696a1d\") " pod="openshift-network-operator/iptables-alerter-tnrwv" Sep 10 03:48:08.933794 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.933514 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/271dd21f-0eed-4858-810a-687acbea64ec-konnectivity-ca\") pod \"konnectivity-agent-sjkng\" (UID: \"271dd21f-0eed-4858-810a-687acbea64ec\") " pod="kube-system/konnectivity-agent-sjkng" Sep 10 03:48:08.933794 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.933536 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1d9fdaf3-79da-4413-8a44-346fe29074be-cnibin\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.933794 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.933552 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1d9fdaf3-79da-4413-8a44-346fe29074be-host-var-lib-cni-multus\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.933794 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.933581 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmczd\" (UniqueName: \"kubernetes.io/projected/1d9fdaf3-79da-4413-8a44-346fe29074be-kube-api-access-wmczd\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.933794 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.933601 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5579d\" (UniqueName: \"kubernetes.io/projected/f0b85b12-5116-477e-86d0-2946bc920ded-kube-api-access-5579d\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.933794 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.933616 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fca9ba6d-4934-47e4-8be5-4296c7a6c530-metrics-certs\") pod \"network-metrics-daemon-htpt9\" (UID: \"fca9ba6d-4934-47e4-8be5-4296c7a6c530\") " pod="openshift-multus/network-metrics-daemon-htpt9" Sep 10 03:48:08.933794 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.933633 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-etc-openvswitch\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.933794 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.933658 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/289045b6-dd39-4301-ac33-3f4fe69ad152-run\") pod \"tuned-kkrg5\" (UID: \"289045b6-dd39-4301-ac33-3f4fe69ad152\") " pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:08.933794 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.933688 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/24e969cd-fc73-4628-bcd8-0abb930691d8-cni-binary-copy\") pod \"multus-additional-cni-plugins-6tbmr\" (UID: \"24e969cd-fc73-4628-bcd8-0abb930691d8\") " pod="openshift-multus/multus-additional-cni-plugins-6tbmr" Sep 10 03:48:08.933794 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.933712 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1d9fdaf3-79da-4413-8a44-346fe29074be-multus-daemon-config\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.934595 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.933737 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f9d9771e-7f4b-4fd5-ac4a-f44b44379f17-registration-dir\") pod \"aws-ebs-csi-driver-node-rrqkg\" (UID: \"f9d9771e-7f4b-4fd5-ac4a-f44b44379f17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rrqkg" Sep 10 03:48:08.934595 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.933763 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/289045b6-dd39-4301-ac33-3f4fe69ad152-sys\") pod \"tuned-kkrg5\" (UID: \"289045b6-dd39-4301-ac33-3f4fe69ad152\") " pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:08.934595 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.933779 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/289045b6-dd39-4301-ac33-3f4fe69ad152-host\") pod \"tuned-kkrg5\" (UID: \"289045b6-dd39-4301-ac33-3f4fe69ad152\") " pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:08.934595 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.933793 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfxkj\" (UniqueName: \"kubernetes.io/projected/29111c2f-bef9-4076-9014-7f041915d6df-kube-api-access-bfxkj\") pod \"node-resolver-7lpdd\" (UID: \"29111c2f-bef9-4076-9014-7f041915d6df\") " pod="openshift-dns/node-resolver-7lpdd" Sep 10 03:48:08.934595 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.933808 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/24e969cd-fc73-4628-bcd8-0abb930691d8-system-cni-dir\") pod \"multus-additional-cni-plugins-6tbmr\" (UID: \"24e969cd-fc73-4628-bcd8-0abb930691d8\") " pod="openshift-multus/multus-additional-cni-plugins-6tbmr" Sep 10 03:48:08.934595 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.933823 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/72f457e8-c3eb-4b62-81e6-10cdd0f78d63-dbus\") pod \"global-pull-secret-syncer-2z94t\" (UID: \"72f457e8-c3eb-4b62-81e6-10cdd0f78d63\") " pod="kube-system/global-pull-secret-syncer-2z94t" Sep 10 03:48:08.934595 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.933840 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f9d9771e-7f4b-4fd5-ac4a-f44b44379f17-device-dir\") pod \"aws-ebs-csi-driver-node-rrqkg\" (UID: \"f9d9771e-7f4b-4fd5-ac4a-f44b44379f17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rrqkg" Sep 10 03:48:08.934595 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.933863 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1d9fdaf3-79da-4413-8a44-346fe29074be-host-var-lib-cni-bin\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.934595 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.933877 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/289045b6-dd39-4301-ac33-3f4fe69ad152-etc-sysctl-d\") pod \"tuned-kkrg5\" (UID: \"289045b6-dd39-4301-ac33-3f4fe69ad152\") " pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:08.934595 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.933891 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/289045b6-dd39-4301-ac33-3f4fe69ad152-tmp\") pod \"tuned-kkrg5\" (UID: \"289045b6-dd39-4301-ac33-3f4fe69ad152\") " pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:08.934595 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.933911 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/24e969cd-fc73-4628-bcd8-0abb930691d8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6tbmr\" (UID: \"24e969cd-fc73-4628-bcd8-0abb930691d8\") " pod="openshift-multus/multus-additional-cni-plugins-6tbmr" Sep 10 03:48:08.934595 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.933933 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1d9fdaf3-79da-4413-8a44-346fe29074be-host-run-netns\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.934595 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.933956 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1d9fdaf3-79da-4413-8a44-346fe29074be-host-run-k8s-cni-cncf-io\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.934595 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.934001 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f9d9771e-7f4b-4fd5-ac4a-f44b44379f17-socket-dir\") pod \"aws-ebs-csi-driver-node-rrqkg\" (UID: \"f9d9771e-7f4b-4fd5-ac4a-f44b44379f17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rrqkg" Sep 10 03:48:08.934595 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.934035 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-run-systemd\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.934595 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.934064 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f0b85b12-5116-477e-86d0-2946bc920ded-ovn-node-metrics-cert\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.934595 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.934084 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2ld7\" (UniqueName: \"kubernetes.io/projected/289045b6-dd39-4301-ac33-3f4fe69ad152-kube-api-access-p2ld7\") pod \"tuned-kkrg5\" (UID: \"289045b6-dd39-4301-ac33-3f4fe69ad152\") " pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:08.935394 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.934107 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/24e969cd-fc73-4628-bcd8-0abb930691d8-cnibin\") pod \"multus-additional-cni-plugins-6tbmr\" (UID: \"24e969cd-fc73-4628-bcd8-0abb930691d8\") " pod="openshift-multus/multus-additional-cni-plugins-6tbmr" Sep 10 03:48:08.935394 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.934124 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttlg7\" (UniqueName: \"kubernetes.io/projected/336788f7-971b-4071-80e6-afaa13696a1d-kube-api-access-ttlg7\") pod \"iptables-alerter-tnrwv\" (UID: \"336788f7-971b-4071-80e6-afaa13696a1d\") " pod="openshift-network-operator/iptables-alerter-tnrwv" Sep 10 03:48:08.935394 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.934154 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1d9fdaf3-79da-4413-8a44-346fe29074be-host-var-lib-kubelet\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.935394 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.934171 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-host-cni-bin\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.935394 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.934195 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-host-cni-netd\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.935394 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.934225 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tfhr\" (UniqueName: \"kubernetes.io/projected/b4c0f4fb-e2d6-4ed5-969e-ec1ff4aaff92-kube-api-access-7tfhr\") pod \"node-ca-r58nd\" (UID: \"b4c0f4fb-e2d6-4ed5-969e-ec1ff4aaff92\") " pod="openshift-image-registry/node-ca-r58nd" Sep 10 03:48:08.935394 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.934245 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f9d9771e-7f4b-4fd5-ac4a-f44b44379f17-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rrqkg\" (UID: \"f9d9771e-7f4b-4fd5-ac4a-f44b44379f17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rrqkg" Sep 10 03:48:08.935394 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.934265 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1d9fdaf3-79da-4413-8a44-346fe29074be-multus-socket-dir-parent\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.935394 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.934286 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1d9fdaf3-79da-4413-8a44-346fe29074be-host-run-multus-certs\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.935394 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.934300 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-log-socket\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.935394 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.934314 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/289045b6-dd39-4301-ac33-3f4fe69ad152-lib-modules\") pod \"tuned-kkrg5\" (UID: \"289045b6-dd39-4301-ac33-3f4fe69ad152\") " pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:08.935394 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.934329 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29111c2f-bef9-4076-9014-7f041915d6df-tmp-dir\") pod \"node-resolver-7lpdd\" (UID: \"29111c2f-bef9-4076-9014-7f041915d6df\") " pod="openshift-dns/node-resolver-7lpdd" Sep 10 03:48:08.935394 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.934350 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/24e969cd-fc73-4628-bcd8-0abb930691d8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6tbmr\" (UID: \"24e969cd-fc73-4628-bcd8-0abb930691d8\") " pod="openshift-multus/multus-additional-cni-plugins-6tbmr" Sep 10 03:48:08.935394 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.934369 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/289045b6-dd39-4301-ac33-3f4fe69ad152-etc-sysconfig\") pod \"tuned-kkrg5\" (UID: \"289045b6-dd39-4301-ac33-3f4fe69ad152\") " pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:08.935394 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.934391 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/289045b6-dd39-4301-ac33-3f4fe69ad152-etc-kubernetes\") pod \"tuned-kkrg5\" (UID: \"289045b6-dd39-4301-ac33-3f4fe69ad152\") " pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:08.935394 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.934416 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/289045b6-dd39-4301-ac33-3f4fe69ad152-var-lib-kubelet\") pod \"tuned-kkrg5\" (UID: \"289045b6-dd39-4301-ac33-3f4fe69ad152\") " pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:08.935394 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.934444 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-systemd-units\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.936217 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.934490 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-host-run-netns\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.936217 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.934519 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f0b85b12-5116-477e-86d0-2946bc920ded-ovnkube-script-lib\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.936217 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.934545 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/289045b6-dd39-4301-ac33-3f4fe69ad152-etc-modprobe-d\") pod \"tuned-kkrg5\" (UID: \"289045b6-dd39-4301-ac33-3f4fe69ad152\") " pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:08.936217 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.934561 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/289045b6-dd39-4301-ac33-3f4fe69ad152-etc-tuned\") pod \"tuned-kkrg5\" (UID: \"289045b6-dd39-4301-ac33-3f4fe69ad152\") " pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:08.936217 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.934576 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1d9fdaf3-79da-4413-8a44-346fe29074be-multus-cni-dir\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.936217 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.934595 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1d9fdaf3-79da-4413-8a44-346fe29074be-cni-binary-copy\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.936217 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.934617 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1d9fdaf3-79da-4413-8a44-346fe29074be-multus-conf-dir\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.936217 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.934650 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f0b85b12-5116-477e-86d0-2946bc920ded-env-overrides\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.936217 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.934667 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/29111c2f-bef9-4076-9014-7f041915d6df-hosts-file\") pod \"node-resolver-7lpdd\" (UID: \"29111c2f-bef9-4076-9014-7f041915d6df\") " pod="openshift-dns/node-resolver-7lpdd" Sep 10 03:48:08.936217 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.934693 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/289045b6-dd39-4301-ac33-3f4fe69ad152-etc-sysctl-conf\") pod \"tuned-kkrg5\" (UID: \"289045b6-dd39-4301-ac33-3f4fe69ad152\") " pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:08.936217 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.934719 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/289045b6-dd39-4301-ac33-3f4fe69ad152-etc-systemd\") pod \"tuned-kkrg5\" (UID: \"289045b6-dd39-4301-ac33-3f4fe69ad152\") " pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:08.936217 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.934741 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f9d9771e-7f4b-4fd5-ac4a-f44b44379f17-etc-selinux\") pod \"aws-ebs-csi-driver-node-rrqkg\" (UID: \"f9d9771e-7f4b-4fd5-ac4a-f44b44379f17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rrqkg" Sep 10 03:48:08.936217 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.934763 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f9d9771e-7f4b-4fd5-ac4a-f44b44379f17-sys-fs\") pod \"aws-ebs-csi-driver-node-rrqkg\" (UID: \"f9d9771e-7f4b-4fd5-ac4a-f44b44379f17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rrqkg" Sep 10 03:48:08.936217 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.934783 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1d9fdaf3-79da-4413-8a44-346fe29074be-etc-kubernetes\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.936217 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.934798 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-host-slash\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.936217 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.934827 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-host-run-ovn-kubernetes\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.936217 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.934860 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/336788f7-971b-4071-80e6-afaa13696a1d-host-slash\") pod \"iptables-alerter-tnrwv\" (UID: \"336788f7-971b-4071-80e6-afaa13696a1d\") " pod="openshift-network-operator/iptables-alerter-tnrwv" Sep 10 03:48:08.936999 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.934877 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-var-lib-openvswitch\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.936999 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.934896 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.936999 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.934928 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f0b85b12-5116-477e-86d0-2946bc920ded-ovnkube-config\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.936999 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.935504 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f0b85b12-5116-477e-86d0-2946bc920ded-ovnkube-config\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.936999 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.935584 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b4c0f4fb-e2d6-4ed5-969e-ec1ff4aaff92-host\") pod \"node-ca-r58nd\" (UID: \"b4c0f4fb-e2d6-4ed5-969e-ec1ff4aaff92\") " pod="openshift-image-registry/node-ca-r58nd" Sep 10 03:48:08.936999 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.936233 2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 10 03:48:08.936999 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.936370 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1d9fdaf3-79da-4413-8a44-346fe29074be-host-var-lib-kubelet\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.936999 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.936417 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-host-cni-bin\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.936999 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.936470 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-host-cni-netd\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.936999 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.936570 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-host-kubelet\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.936999 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.936627 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-run-ovn\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.936999 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.936687 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f9d9771e-7f4b-4fd5-ac4a-f44b44379f17-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rrqkg\" (UID: \"f9d9771e-7f4b-4fd5-ac4a-f44b44379f17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rrqkg" Sep 10 03:48:08.936999 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.936733 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-node-log\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.936999 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.936790 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1d9fdaf3-79da-4413-8a44-346fe29074be-multus-socket-dir-parent\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.936999 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.936834 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1d9fdaf3-79da-4413-8a44-346fe29074be-host-run-multus-certs\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.936999 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.936876 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-log-socket\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.937783 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.937013 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b4c0f4fb-e2d6-4ed5-969e-ec1ff4aaff92-serviceca\") pod \"node-ca-r58nd\" (UID: \"b4c0f4fb-e2d6-4ed5-969e-ec1ff4aaff92\") " pod="openshift-image-registry/node-ca-r58nd" Sep 10 03:48:08.937783 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.937045 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/289045b6-dd39-4301-ac33-3f4fe69ad152-lib-modules\") pod \"tuned-kkrg5\" (UID: \"289045b6-dd39-4301-ac33-3f4fe69ad152\") " pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:08.937783 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.937115 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1d9fdaf3-79da-4413-8a44-346fe29074be-system-cni-dir\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.937783 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.937161 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f9d9771e-7f4b-4fd5-ac4a-f44b44379f17-registration-dir\") pod \"aws-ebs-csi-driver-node-rrqkg\" (UID: \"f9d9771e-7f4b-4fd5-ac4a-f44b44379f17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rrqkg" Sep 10 03:48:08.937783 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.937203 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/289045b6-dd39-4301-ac33-3f4fe69ad152-sys\") pod \"tuned-kkrg5\" (UID: \"289045b6-dd39-4301-ac33-3f4fe69ad152\") " pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:08.937783 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.937239 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/289045b6-dd39-4301-ac33-3f4fe69ad152-host\") pod \"tuned-kkrg5\" (UID: \"289045b6-dd39-4301-ac33-3f4fe69ad152\") " pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:08.937783 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.937454 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f9d9771e-7f4b-4fd5-ac4a-f44b44379f17-device-dir\") pod \"aws-ebs-csi-driver-node-rrqkg\" (UID: \"f9d9771e-7f4b-4fd5-ac4a-f44b44379f17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rrqkg" Sep 10 03:48:08.937783 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.937490 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1d9fdaf3-79da-4413-8a44-346fe29074be-host-var-lib-cni-bin\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.937783 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.937515 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1d9fdaf3-79da-4413-8a44-346fe29074be-multus-daemon-config\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.937783 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.937567 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1d9fdaf3-79da-4413-8a44-346fe29074be-hostroot\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.937783 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.937568 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/289045b6-dd39-4301-ac33-3f4fe69ad152-etc-sysctl-d\") pod \"tuned-kkrg5\" (UID: \"289045b6-dd39-4301-ac33-3f4fe69ad152\") " pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:08.937783 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.937613 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-run-openvswitch\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.937783 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.937629 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1d9fdaf3-79da-4413-8a44-346fe29074be-host-run-netns\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.937783 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.937662 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1d9fdaf3-79da-4413-8a44-346fe29074be-host-run-k8s-cni-cncf-io\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.937783 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.937753 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f9d9771e-7f4b-4fd5-ac4a-f44b44379f17-socket-dir\") pod \"aws-ebs-csi-driver-node-rrqkg\" (UID: \"f9d9771e-7f4b-4fd5-ac4a-f44b44379f17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rrqkg" Sep 10 03:48:08.937783 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.937794 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-run-systemd\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.938527 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.938013 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1d9fdaf3-79da-4413-8a44-346fe29074be-multus-conf-dir\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.938527 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.938180 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f9d9771e-7f4b-4fd5-ac4a-f44b44379f17-sys-fs\") pod \"aws-ebs-csi-driver-node-rrqkg\" (UID: \"f9d9771e-7f4b-4fd5-ac4a-f44b44379f17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rrqkg" Sep 10 03:48:08.938527 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.938523 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/289045b6-dd39-4301-ac33-3f4fe69ad152-etc-sysconfig\") pod \"tuned-kkrg5\" (UID: \"289045b6-dd39-4301-ac33-3f4fe69ad152\") " pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:08.938672 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.938398 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29111c2f-bef9-4076-9014-7f041915d6df-tmp-dir\") pod \"node-resolver-7lpdd\" (UID: \"29111c2f-bef9-4076-9014-7f041915d6df\") " pod="openshift-dns/node-resolver-7lpdd" Sep 10 03:48:08.938672 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.938566 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/289045b6-dd39-4301-ac33-3f4fe69ad152-etc-sysctl-conf\") pod \"tuned-kkrg5\" (UID: \"289045b6-dd39-4301-ac33-3f4fe69ad152\") " pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:08.938672 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.938581 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/289045b6-dd39-4301-ac33-3f4fe69ad152-etc-systemd\") pod \"tuned-kkrg5\" (UID: \"289045b6-dd39-4301-ac33-3f4fe69ad152\") " pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:08.938672 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.938620 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/29111c2f-bef9-4076-9014-7f041915d6df-hosts-file\") pod \"node-resolver-7lpdd\" (UID: \"29111c2f-bef9-4076-9014-7f041915d6df\") " pod="openshift-dns/node-resolver-7lpdd" Sep 10 03:48:08.938672 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.938625 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/289045b6-dd39-4301-ac33-3f4fe69ad152-etc-kubernetes\") pod \"tuned-kkrg5\" (UID: \"289045b6-dd39-4301-ac33-3f4fe69ad152\") " pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:08.938909 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.938670 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f9d9771e-7f4b-4fd5-ac4a-f44b44379f17-etc-selinux\") pod \"aws-ebs-csi-driver-node-rrqkg\" (UID: \"f9d9771e-7f4b-4fd5-ac4a-f44b44379f17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rrqkg" Sep 10 03:48:08.938909 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.938681 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/289045b6-dd39-4301-ac33-3f4fe69ad152-var-lib-kubelet\") pod \"tuned-kkrg5\" (UID: \"289045b6-dd39-4301-ac33-3f4fe69ad152\") " pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:08.938909 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.938673 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1d9fdaf3-79da-4413-8a44-346fe29074be-host-var-lib-cni-multus\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.938909 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.938713 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f0b85b12-5116-477e-86d0-2946bc920ded-env-overrides\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.938909 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.938727 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-host-run-netns\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.938909 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.938734 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-systemd-units\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.938909 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.938803 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-host-run-ovn-kubernetes\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.940275 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.939035 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1d9fdaf3-79da-4413-8a44-346fe29074be-etc-kubernetes\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.940275 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.939084 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-host-slash\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.940275 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.939103 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/289045b6-dd39-4301-ac33-3f4fe69ad152-etc-modprobe-d\") pod \"tuned-kkrg5\" (UID: \"289045b6-dd39-4301-ac33-3f4fe69ad152\") " pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:08.940275 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.939258 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/271dd21f-0eed-4858-810a-687acbea64ec-konnectivity-ca\") pod \"konnectivity-agent-sjkng\" (UID: \"271dd21f-0eed-4858-810a-687acbea64ec\") " pod="kube-system/konnectivity-agent-sjkng" Sep 10 03:48:08.940275 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.939328 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1d9fdaf3-79da-4413-8a44-346fe29074be-cnibin\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.940275 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.939350 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f0b85b12-5116-477e-86d0-2946bc920ded-ovnkube-script-lib\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.940275 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.939401 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-var-lib-openvswitch\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.940275 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.939426 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1d9fdaf3-79da-4413-8a44-346fe29074be-multus-cni-dir\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.940275 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.939460 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/289045b6-dd39-4301-ac33-3f4fe69ad152-run\") pod \"tuned-kkrg5\" (UID: \"289045b6-dd39-4301-ac33-3f4fe69ad152\") " pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:08.940275 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.939509 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.940275 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.939635 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f0b85b12-5116-477e-86d0-2946bc920ded-etc-openvswitch\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.940275 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.939898 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1d9fdaf3-79da-4413-8a44-346fe29074be-cni-binary-copy\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.940728 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.940572 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f0b85b12-5116-477e-86d0-2946bc920ded-ovn-node-metrics-cert\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:08.940910 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.940891 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/271dd21f-0eed-4858-810a-687acbea64ec-agent-certs\") pod \"konnectivity-agent-sjkng\" (UID: \"271dd21f-0eed-4858-810a-687acbea64ec\") " pod="kube-system/konnectivity-agent-sjkng" Sep 10 03:48:08.941207 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.941189 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/289045b6-dd39-4301-ac33-3f4fe69ad152-tmp\") pod \"tuned-kkrg5\" (UID: \"289045b6-dd39-4301-ac33-3f4fe69ad152\") " pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:08.942669 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.942572 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/289045b6-dd39-4301-ac33-3f4fe69ad152-etc-tuned\") pod \"tuned-kkrg5\" (UID: \"289045b6-dd39-4301-ac33-3f4fe69ad152\") " pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:08.944436 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.944386 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-77.ec2.internal" event={"ID":"81cf177ab4229900eb060a4c89e0a554","Type":"ContainerStarted","Data":"79501ce27c9dc7227dc933521d3c38fbdd5d37326e4751e9c137abc17dbac1a3"} Sep 10 03:48:08.945733 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.945369 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tfhr\" (UniqueName: \"kubernetes.io/projected/b4c0f4fb-e2d6-4ed5-969e-ec1ff4aaff92-kube-api-access-7tfhr\") pod \"node-ca-r58nd\" (UID: \"b4c0f4fb-e2d6-4ed5-969e-ec1ff4aaff92\") " pod="openshift-image-registry/node-ca-r58nd" Sep 10 03:48:08.945733 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.945664 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kt8rf\" (UniqueName: \"kubernetes.io/projected/f9d9771e-7f4b-4fd5-ac4a-f44b44379f17-kube-api-access-kt8rf\") pod \"aws-ebs-csi-driver-node-rrqkg\" (UID: \"f9d9771e-7f4b-4fd5-ac4a-f44b44379f17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rrqkg" Sep 10 03:48:08.945889 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.945870 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2ld7\" (UniqueName: \"kubernetes.io/projected/289045b6-dd39-4301-ac33-3f4fe69ad152-kube-api-access-p2ld7\") pod \"tuned-kkrg5\" (UID: \"289045b6-dd39-4301-ac33-3f4fe69ad152\") " pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:08.945946 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.945922 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-77.ec2.internal" event={"ID":"6524cf98e47d7fba2fecd528cfc30ded","Type":"ContainerStarted","Data":"975831ef3f37d69b3fac7170df9851c711dc66b97e7bed2f6cc89a8a27a73f4e"} Sep 10 03:48:08.947015 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.946992 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfxkj\" (UniqueName: \"kubernetes.io/projected/29111c2f-bef9-4076-9014-7f041915d6df-kube-api-access-bfxkj\") pod \"node-resolver-7lpdd\" (UID: \"29111c2f-bef9-4076-9014-7f041915d6df\") " pod="openshift-dns/node-resolver-7lpdd" Sep 10 03:48:08.948545 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.948525 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmczd\" (UniqueName: \"kubernetes.io/projected/1d9fdaf3-79da-4413-8a44-346fe29074be-kube-api-access-wmczd\") pod \"multus-wnlbc\" (UID: \"1d9fdaf3-79da-4413-8a44-346fe29074be\") " pod="openshift-multus/multus-wnlbc" Sep 10 03:48:08.948653 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:08.948631 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5579d\" (UniqueName: \"kubernetes.io/projected/f0b85b12-5116-477e-86d0-2946bc920ded-kube-api-access-5579d\") pod \"ovnkube-node-cqmkb\" (UID: \"f0b85b12-5116-477e-86d0-2946bc920ded\") " pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:09.035745 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.035718 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/336788f7-971b-4071-80e6-afaa13696a1d-host-slash\") pod \"iptables-alerter-tnrwv\" (UID: \"336788f7-971b-4071-80e6-afaa13696a1d\") " pod="openshift-network-operator/iptables-alerter-tnrwv" Sep 10 03:48:09.035915 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.035757 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/24e969cd-fc73-4628-bcd8-0abb930691d8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6tbmr\" (UID: \"24e969cd-fc73-4628-bcd8-0abb930691d8\") " pod="openshift-multus/multus-additional-cni-plugins-6tbmr" Sep 10 03:48:09.035915 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.035782 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/72f457e8-c3eb-4b62-81e6-10cdd0f78d63-original-pull-secret\") pod \"global-pull-secret-syncer-2z94t\" (UID: \"72f457e8-c3eb-4b62-81e6-10cdd0f78d63\") " pod="kube-system/global-pull-secret-syncer-2z94t" Sep 10 03:48:09.035915 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.035803 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjq6z\" (UniqueName: \"kubernetes.io/projected/fca9ba6d-4934-47e4-8be5-4296c7a6c530-kube-api-access-fjq6z\") pod \"network-metrics-daemon-htpt9\" (UID: \"fca9ba6d-4934-47e4-8be5-4296c7a6c530\") " pod="openshift-multus/network-metrics-daemon-htpt9" Sep 10 03:48:09.035915 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.035831 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdjfj\" (UniqueName: \"kubernetes.io/projected/24e969cd-fc73-4628-bcd8-0abb930691d8-kube-api-access-hdjfj\") pod \"multus-additional-cni-plugins-6tbmr\" (UID: \"24e969cd-fc73-4628-bcd8-0abb930691d8\") " pod="openshift-multus/multus-additional-cni-plugins-6tbmr" Sep 10 03:48:09.035915 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.035852 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/72f457e8-c3eb-4b62-81e6-10cdd0f78d63-kubelet-config\") pod \"global-pull-secret-syncer-2z94t\" (UID: \"72f457e8-c3eb-4b62-81e6-10cdd0f78d63\") " pod="kube-system/global-pull-secret-syncer-2z94t" Sep 10 03:48:09.035915 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.035857 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/336788f7-971b-4071-80e6-afaa13696a1d-host-slash\") pod \"iptables-alerter-tnrwv\" (UID: \"336788f7-971b-4071-80e6-afaa13696a1d\") " pod="openshift-network-operator/iptables-alerter-tnrwv" Sep 10 03:48:09.035915 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.035873 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc9lh\" (UniqueName: \"kubernetes.io/projected/6be9cc5c-8c3c-4985-98ab-ec0848534ff0-kube-api-access-hc9lh\") pod \"network-check-target-7r9qf\" (UID: \"6be9cc5c-8c3c-4985-98ab-ec0848534ff0\") " pod="openshift-network-diagnostics/network-check-target-7r9qf" Sep 10 03:48:09.036260 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.035918 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/336788f7-971b-4071-80e6-afaa13696a1d-iptables-alerter-script\") pod \"iptables-alerter-tnrwv\" (UID: \"336788f7-971b-4071-80e6-afaa13696a1d\") " pod="openshift-network-operator/iptables-alerter-tnrwv" Sep 10 03:48:09.036260 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.035960 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fca9ba6d-4934-47e4-8be5-4296c7a6c530-metrics-certs\") pod \"network-metrics-daemon-htpt9\" (UID: \"fca9ba6d-4934-47e4-8be5-4296c7a6c530\") " pod="openshift-multus/network-metrics-daemon-htpt9" Sep 10 03:48:09.036260 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.036007 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/24e969cd-fc73-4628-bcd8-0abb930691d8-cni-binary-copy\") pod \"multus-additional-cni-plugins-6tbmr\" (UID: \"24e969cd-fc73-4628-bcd8-0abb930691d8\") " pod="openshift-multus/multus-additional-cni-plugins-6tbmr" Sep 10 03:48:09.036260 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.036040 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/24e969cd-fc73-4628-bcd8-0abb930691d8-system-cni-dir\") pod \"multus-additional-cni-plugins-6tbmr\" (UID: \"24e969cd-fc73-4628-bcd8-0abb930691d8\") " pod="openshift-multus/multus-additional-cni-plugins-6tbmr" Sep 10 03:48:09.036260 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.036066 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/72f457e8-c3eb-4b62-81e6-10cdd0f78d63-dbus\") pod \"global-pull-secret-syncer-2z94t\" (UID: \"72f457e8-c3eb-4b62-81e6-10cdd0f78d63\") " pod="kube-system/global-pull-secret-syncer-2z94t" Sep 10 03:48:09.036260 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.036097 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/24e969cd-fc73-4628-bcd8-0abb930691d8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6tbmr\" (UID: \"24e969cd-fc73-4628-bcd8-0abb930691d8\") " pod="openshift-multus/multus-additional-cni-plugins-6tbmr" Sep 10 03:48:09.036260 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.036130 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/24e969cd-fc73-4628-bcd8-0abb930691d8-cnibin\") pod \"multus-additional-cni-plugins-6tbmr\" (UID: \"24e969cd-fc73-4628-bcd8-0abb930691d8\") " pod="openshift-multus/multus-additional-cni-plugins-6tbmr" Sep 10 03:48:09.036260 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.036155 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttlg7\" (UniqueName: \"kubernetes.io/projected/336788f7-971b-4071-80e6-afaa13696a1d-kube-api-access-ttlg7\") pod \"iptables-alerter-tnrwv\" (UID: \"336788f7-971b-4071-80e6-afaa13696a1d\") " pod="openshift-network-operator/iptables-alerter-tnrwv" Sep 10 03:48:09.036260 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.036193 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/24e969cd-fc73-4628-bcd8-0abb930691d8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6tbmr\" (UID: \"24e969cd-fc73-4628-bcd8-0abb930691d8\") " pod="openshift-multus/multus-additional-cni-plugins-6tbmr" Sep 10 03:48:09.036693 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:09.036270 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 03:48:09.036693 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:09.036343 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72f457e8-c3eb-4b62-81e6-10cdd0f78d63-original-pull-secret podName:72f457e8-c3eb-4b62-81e6-10cdd0f78d63 nodeName:}" failed. No retries permitted until 2026-09-10 03:48:09.536314072 +0000 UTC m=+3.061693170 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/72f457e8-c3eb-4b62-81e6-10cdd0f78d63-original-pull-secret") pod "global-pull-secret-syncer-2z94t" (UID: "72f457e8-c3eb-4b62-81e6-10cdd0f78d63") : object "kube-system"/"original-pull-secret" not registered Sep 10 03:48:09.036693 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.036372 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/24e969cd-fc73-4628-bcd8-0abb930691d8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6tbmr\" (UID: \"24e969cd-fc73-4628-bcd8-0abb930691d8\") " pod="openshift-multus/multus-additional-cni-plugins-6tbmr" Sep 10 03:48:09.036693 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.036407 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/24e969cd-fc73-4628-bcd8-0abb930691d8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6tbmr\" (UID: \"24e969cd-fc73-4628-bcd8-0abb930691d8\") " pod="openshift-multus/multus-additional-cni-plugins-6tbmr" Sep 10 03:48:09.036693 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.036575 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/72f457e8-c3eb-4b62-81e6-10cdd0f78d63-kubelet-config\") pod \"global-pull-secret-syncer-2z94t\" (UID: \"72f457e8-c3eb-4b62-81e6-10cdd0f78d63\") " pod="kube-system/global-pull-secret-syncer-2z94t" Sep 10 03:48:09.036935 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:09.036699 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 03:48:09.036935 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:09.036789 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fca9ba6d-4934-47e4-8be5-4296c7a6c530-metrics-certs podName:fca9ba6d-4934-47e4-8be5-4296c7a6c530 nodeName:}" failed. No retries permitted until 2026-09-10 03:48:09.536772362 +0000 UTC m=+3.062151466 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fca9ba6d-4934-47e4-8be5-4296c7a6c530-metrics-certs") pod "network-metrics-daemon-htpt9" (UID: "fca9ba6d-4934-47e4-8be5-4296c7a6c530") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 03:48:09.036935 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.036819 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/72f457e8-c3eb-4b62-81e6-10cdd0f78d63-dbus\") pod \"global-pull-secret-syncer-2z94t\" (UID: \"72f457e8-c3eb-4b62-81e6-10cdd0f78d63\") " pod="kube-system/global-pull-secret-syncer-2z94t" Sep 10 03:48:09.036935 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.036869 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/24e969cd-fc73-4628-bcd8-0abb930691d8-cnibin\") pod \"multus-additional-cni-plugins-6tbmr\" (UID: \"24e969cd-fc73-4628-bcd8-0abb930691d8\") " pod="openshift-multus/multus-additional-cni-plugins-6tbmr" Sep 10 03:48:09.037152 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.037068 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/336788f7-971b-4071-80e6-afaa13696a1d-iptables-alerter-script\") pod \"iptables-alerter-tnrwv\" (UID: \"336788f7-971b-4071-80e6-afaa13696a1d\") " pod="openshift-network-operator/iptables-alerter-tnrwv" Sep 10 03:48:09.037289 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.037267 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/24e969cd-fc73-4628-bcd8-0abb930691d8-cni-binary-copy\") pod \"multus-additional-cni-plugins-6tbmr\" (UID: \"24e969cd-fc73-4628-bcd8-0abb930691d8\") " pod="openshift-multus/multus-additional-cni-plugins-6tbmr" Sep 10 03:48:09.037350 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.037331 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/24e969cd-fc73-4628-bcd8-0abb930691d8-system-cni-dir\") pod \"multus-additional-cni-plugins-6tbmr\" (UID: \"24e969cd-fc73-4628-bcd8-0abb930691d8\") " pod="openshift-multus/multus-additional-cni-plugins-6tbmr" Sep 10 03:48:09.037402 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.037364 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/24e969cd-fc73-4628-bcd8-0abb930691d8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6tbmr\" (UID: \"24e969cd-fc73-4628-bcd8-0abb930691d8\") " pod="openshift-multus/multus-additional-cni-plugins-6tbmr" Sep 10 03:48:09.044690 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:09.044656 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 03:48:09.044690 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:09.044692 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 03:48:09.044847 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:09.044702 2581 projected.go:196] Error preparing data for projected volume kube-api-access-hc9lh for pod openshift-network-diagnostics/network-check-target-7r9qf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 03:48:09.044847 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:09.044758 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6be9cc5c-8c3c-4985-98ab-ec0848534ff0-kube-api-access-hc9lh podName:6be9cc5c-8c3c-4985-98ab-ec0848534ff0 nodeName:}" failed. No retries permitted until 2026-09-10 03:48:09.544743914 +0000 UTC m=+3.070123014 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-hc9lh" (UniqueName: "kubernetes.io/projected/6be9cc5c-8c3c-4985-98ab-ec0848534ff0-kube-api-access-hc9lh") pod "network-check-target-7r9qf" (UID: "6be9cc5c-8c3c-4985-98ab-ec0848534ff0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 03:48:09.046923 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.046897 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdjfj\" (UniqueName: \"kubernetes.io/projected/24e969cd-fc73-4628-bcd8-0abb930691d8-kube-api-access-hdjfj\") pod \"multus-additional-cni-plugins-6tbmr\" (UID: \"24e969cd-fc73-4628-bcd8-0abb930691d8\") " pod="openshift-multus/multus-additional-cni-plugins-6tbmr" Sep 10 03:48:09.047127 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.047111 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttlg7\" (UniqueName: \"kubernetes.io/projected/336788f7-971b-4071-80e6-afaa13696a1d-kube-api-access-ttlg7\") pod \"iptables-alerter-tnrwv\" (UID: \"336788f7-971b-4071-80e6-afaa13696a1d\") " pod="openshift-network-operator/iptables-alerter-tnrwv" Sep 10 03:48:09.047457 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.047436 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjq6z\" (UniqueName: \"kubernetes.io/projected/fca9ba6d-4934-47e4-8be5-4296c7a6c530-kube-api-access-fjq6z\") pod \"network-metrics-daemon-htpt9\" (UID: \"fca9ba6d-4934-47e4-8be5-4296c7a6c530\") " pod="openshift-multus/network-metrics-daemon-htpt9" Sep 10 03:48:09.128831 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.128784 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wnlbc" Sep 10 03:48:09.138616 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.138597 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-sjkng" Sep 10 03:48:09.147426 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.147403 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rrqkg" Sep 10 03:48:09.152238 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.152217 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:09.158820 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.158803 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" Sep 10 03:48:09.165272 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.165257 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7lpdd" Sep 10 03:48:09.170881 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.170858 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-r58nd" Sep 10 03:48:09.178664 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.178644 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6tbmr" Sep 10 03:48:09.183159 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.183142 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tnrwv" Sep 10 03:48:09.283854 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.283819 2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 03:48:09.539023 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.538987 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/72f457e8-c3eb-4b62-81e6-10cdd0f78d63-original-pull-secret\") pod \"global-pull-secret-syncer-2z94t\" (UID: \"72f457e8-c3eb-4b62-81e6-10cdd0f78d63\") " pod="kube-system/global-pull-secret-syncer-2z94t" Sep 10 03:48:09.539202 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.539044 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fca9ba6d-4934-47e4-8be5-4296c7a6c530-metrics-certs\") pod \"network-metrics-daemon-htpt9\" (UID: \"fca9ba6d-4934-47e4-8be5-4296c7a6c530\") " pod="openshift-multus/network-metrics-daemon-htpt9" Sep 10 03:48:09.539202 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:09.539134 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 03:48:09.539202 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:09.539164 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 03:48:09.539202 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:09.539204 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72f457e8-c3eb-4b62-81e6-10cdd0f78d63-original-pull-secret podName:72f457e8-c3eb-4b62-81e6-10cdd0f78d63 nodeName:}" failed. No retries permitted until 2026-09-10 03:48:10.539188457 +0000 UTC m=+4.064567540 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/72f457e8-c3eb-4b62-81e6-10cdd0f78d63-original-pull-secret") pod "global-pull-secret-syncer-2z94t" (UID: "72f457e8-c3eb-4b62-81e6-10cdd0f78d63") : object "kube-system"/"original-pull-secret" not registered Sep 10 03:48:09.539418 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:09.539219 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fca9ba6d-4934-47e4-8be5-4296c7a6c530-metrics-certs podName:fca9ba6d-4934-47e4-8be5-4296c7a6c530 nodeName:}" failed. No retries permitted until 2026-09-10 03:48:10.539211589 +0000 UTC m=+4.064590672 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fca9ba6d-4934-47e4-8be5-4296c7a6c530-metrics-certs") pod "network-metrics-daemon-htpt9" (UID: "fca9ba6d-4934-47e4-8be5-4296c7a6c530") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 03:48:09.639492 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.639458 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc9lh\" (UniqueName: \"kubernetes.io/projected/6be9cc5c-8c3c-4985-98ab-ec0848534ff0-kube-api-access-hc9lh\") pod \"network-check-target-7r9qf\" (UID: \"6be9cc5c-8c3c-4985-98ab-ec0848534ff0\") " pod="openshift-network-diagnostics/network-check-target-7r9qf" Sep 10 03:48:09.639667 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:09.639629 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 03:48:09.639667 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:09.639649 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 03:48:09.639667 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:09.639658 2581 projected.go:196] Error preparing data for projected volume kube-api-access-hc9lh for pod openshift-network-diagnostics/network-check-target-7r9qf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 03:48:09.639825 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:09.639715 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6be9cc5c-8c3c-4985-98ab-ec0848534ff0-kube-api-access-hc9lh podName:6be9cc5c-8c3c-4985-98ab-ec0848534ff0 nodeName:}" failed. No retries permitted until 2026-09-10 03:48:10.639698577 +0000 UTC m=+4.165077662 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-hc9lh" (UniqueName: "kubernetes.io/projected/6be9cc5c-8c3c-4985-98ab-ec0848534ff0-kube-api-access-hc9lh") pod "network-check-target-7r9qf" (UID: "6be9cc5c-8c3c-4985-98ab-ec0848534ff0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 03:48:09.719188 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:09.719143 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod289045b6_dd39_4301_ac33_3f4fe69ad152.slice/crio-14865c9ae669519f5967a8219d631f41b2b7519004e89cd89e5279f4d3812827 WatchSource:0}: Error finding container 14865c9ae669519f5967a8219d631f41b2b7519004e89cd89e5279f4d3812827: Status 404 returned error can't find the container with id 14865c9ae669519f5967a8219d631f41b2b7519004e89cd89e5279f4d3812827 Sep 10 03:48:09.720761 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:09.720734 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24e969cd_fc73_4628_bcd8_0abb930691d8.slice/crio-1f375340e068d8c2a395c7c70fbeadf4954fa9d2f7895b31a9bb96fb29846d0a WatchSource:0}: Error finding container 1f375340e068d8c2a395c7c70fbeadf4954fa9d2f7895b31a9bb96fb29846d0a: Status 404 returned error can't find the container with id 1f375340e068d8c2a395c7c70fbeadf4954fa9d2f7895b31a9bb96fb29846d0a Sep 10 03:48:09.721034 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:09.721017 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9d9771e_7f4b_4fd5_ac4a_f44b44379f17.slice/crio-fcf57bd03cf9795e0a928908c547e3dfed5b1bd33019aa71de25bf1974fcdbba WatchSource:0}: Error finding container fcf57bd03cf9795e0a928908c547e3dfed5b1bd33019aa71de25bf1974fcdbba: Status 404 returned error can't find the container with id fcf57bd03cf9795e0a928908c547e3dfed5b1bd33019aa71de25bf1974fcdbba Sep 10 03:48:09.729231 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:09.728736 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod271dd21f_0eed_4858_810a_687acbea64ec.slice/crio-6c3121130a6345bbe4e6e5a2606c27b4d6862ba05aec5bc2a51ea322f0239461 WatchSource:0}: Error finding container 6c3121130a6345bbe4e6e5a2606c27b4d6862ba05aec5bc2a51ea322f0239461: Status 404 returned error can't find the container with id 6c3121130a6345bbe4e6e5a2606c27b4d6862ba05aec5bc2a51ea322f0239461 Sep 10 03:48:09.730987 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:09.730949 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29111c2f_bef9_4076_9014_7f041915d6df.slice/crio-ab7b4ba000c33423e2cff92e5be143bc17558e5cdea0fc455dfad0eb3baf108f WatchSource:0}: Error finding container ab7b4ba000c33423e2cff92e5be143bc17558e5cdea0fc455dfad0eb3baf108f: Status 404 returned error can't find the container with id ab7b4ba000c33423e2cff92e5be143bc17558e5cdea0fc455dfad0eb3baf108f Sep 10 03:48:09.868314 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.868208 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 03:43:08 +0000 UTC" deadline="2028-04-11 21:07:44.630999047 +0000 UTC" Sep 10 03:48:09.868314 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.868263 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13913h19m34.762738258s" Sep 10 03:48:09.950110 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.950079 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-77.ec2.internal" event={"ID":"81cf177ab4229900eb060a4c89e0a554","Type":"ContainerStarted","Data":"e4741a8806972307e801b7f7e8f6c3225fe01644c298668cddc13a55baa672be"} Sep 10 03:48:09.951542 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.951515 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wnlbc" event={"ID":"1d9fdaf3-79da-4413-8a44-346fe29074be","Type":"ContainerStarted","Data":"3ee84f6dd0c96e399a68acb4094bfe36ef8da0d90f7c00ba8182f830b27f8836"} Sep 10 03:48:09.952884 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.952861 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-sjkng" event={"ID":"271dd21f-0eed-4858-810a-687acbea64ec","Type":"ContainerStarted","Data":"6c3121130a6345bbe4e6e5a2606c27b4d6862ba05aec5bc2a51ea322f0239461"} Sep 10 03:48:09.954027 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.954003 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7lpdd" event={"ID":"29111c2f-bef9-4076-9014-7f041915d6df","Type":"ContainerStarted","Data":"ab7b4ba000c33423e2cff92e5be143bc17558e5cdea0fc455dfad0eb3baf108f"} Sep 10 03:48:09.955155 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.955131 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-r58nd" event={"ID":"b4c0f4fb-e2d6-4ed5-969e-ec1ff4aaff92","Type":"ContainerStarted","Data":"dfdb2bcd08f4ebcea7d60a40408d449fbc077acc55a7b5ea88bf5fb69ad24685"} Sep 10 03:48:09.956356 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.956335 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" event={"ID":"f0b85b12-5116-477e-86d0-2946bc920ded","Type":"ContainerStarted","Data":"f7c8aba63760d4b80a06a0923b3b10741b9860b5fcafed3817db4eb4376fd5b7"} Sep 10 03:48:09.957478 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.957450 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rrqkg" event={"ID":"f9d9771e-7f4b-4fd5-ac4a-f44b44379f17","Type":"ContainerStarted","Data":"fcf57bd03cf9795e0a928908c547e3dfed5b1bd33019aa71de25bf1974fcdbba"} Sep 10 03:48:09.958543 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.958527 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tnrwv" event={"ID":"336788f7-971b-4071-80e6-afaa13696a1d","Type":"ContainerStarted","Data":"b3c1f3bd58920a3de66143556e17d475fedf2f4ad487c67e73d1781f685ab7b5"} Sep 10 03:48:09.959578 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.959555 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6tbmr" event={"ID":"24e969cd-fc73-4628-bcd8-0abb930691d8","Type":"ContainerStarted","Data":"1f375340e068d8c2a395c7c70fbeadf4954fa9d2f7895b31a9bb96fb29846d0a"} Sep 10 03:48:09.960495 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.960475 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" event={"ID":"289045b6-dd39-4301-ac33-3f4fe69ad152","Type":"ContainerStarted","Data":"14865c9ae669519f5967a8219d631f41b2b7519004e89cd89e5279f4d3812827"} Sep 10 03:48:09.963852 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:09.963806 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-77.ec2.internal" podStartSLOduration=1.963788868 podStartE2EDuration="1.963788868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 03:48:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:48:09.962882927 +0000 UTC m=+3.488262026" watchObservedRunningTime="2026-09-10 03:48:09.963788868 +0000 UTC m=+3.489167975" Sep 10 03:48:10.546903 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:10.546865 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/72f457e8-c3eb-4b62-81e6-10cdd0f78d63-original-pull-secret\") pod \"global-pull-secret-syncer-2z94t\" (UID: \"72f457e8-c3eb-4b62-81e6-10cdd0f78d63\") " pod="kube-system/global-pull-secret-syncer-2z94t" Sep 10 03:48:10.547092 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:10.546930 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fca9ba6d-4934-47e4-8be5-4296c7a6c530-metrics-certs\") pod \"network-metrics-daemon-htpt9\" (UID: \"fca9ba6d-4934-47e4-8be5-4296c7a6c530\") " pod="openshift-multus/network-metrics-daemon-htpt9" Sep 10 03:48:10.547092 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:10.547071 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 03:48:10.547243 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:10.547133 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fca9ba6d-4934-47e4-8be5-4296c7a6c530-metrics-certs podName:fca9ba6d-4934-47e4-8be5-4296c7a6c530 nodeName:}" failed. No retries permitted until 2026-09-10 03:48:12.547114296 +0000 UTC m=+6.072493385 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fca9ba6d-4934-47e4-8be5-4296c7a6c530-metrics-certs") pod "network-metrics-daemon-htpt9" (UID: "fca9ba6d-4934-47e4-8be5-4296c7a6c530") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 03:48:10.547513 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:10.547492 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 03:48:10.547593 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:10.547552 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72f457e8-c3eb-4b62-81e6-10cdd0f78d63-original-pull-secret podName:72f457e8-c3eb-4b62-81e6-10cdd0f78d63 nodeName:}" failed. No retries permitted until 2026-09-10 03:48:12.547538112 +0000 UTC m=+6.072917197 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/72f457e8-c3eb-4b62-81e6-10cdd0f78d63-original-pull-secret") pod "global-pull-secret-syncer-2z94t" (UID: "72f457e8-c3eb-4b62-81e6-10cdd0f78d63") : object "kube-system"/"original-pull-secret" not registered Sep 10 03:48:10.647766 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:10.647729 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc9lh\" (UniqueName: \"kubernetes.io/projected/6be9cc5c-8c3c-4985-98ab-ec0848534ff0-kube-api-access-hc9lh\") pod \"network-check-target-7r9qf\" (UID: \"6be9cc5c-8c3c-4985-98ab-ec0848534ff0\") " pod="openshift-network-diagnostics/network-check-target-7r9qf" Sep 10 03:48:10.647924 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:10.647906 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 03:48:10.648029 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:10.647925 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 03:48:10.648029 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:10.647939 2581 projected.go:196] Error preparing data for projected volume kube-api-access-hc9lh for pod openshift-network-diagnostics/network-check-target-7r9qf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 03:48:10.648029 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:10.648018 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6be9cc5c-8c3c-4985-98ab-ec0848534ff0-kube-api-access-hc9lh podName:6be9cc5c-8c3c-4985-98ab-ec0848534ff0 nodeName:}" failed. No retries permitted until 2026-09-10 03:48:12.647999378 +0000 UTC m=+6.173378472 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-hc9lh" (UniqueName: "kubernetes.io/projected/6be9cc5c-8c3c-4985-98ab-ec0848534ff0-kube-api-access-hc9lh") pod "network-check-target-7r9qf" (UID: "6be9cc5c-8c3c-4985-98ab-ec0848534ff0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 03:48:10.869158 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:10.869067 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 03:43:08 +0000 UTC" deadline="2028-02-05 06:27:57.487666017 +0000 UTC" Sep 10 03:48:10.869158 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:10.869109 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12314h39m46.618561061s" Sep 10 03:48:10.950077 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:10.950048 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7r9qf" Sep 10 03:48:10.950250 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:10.950201 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7r9qf" podUID="6be9cc5c-8c3c-4985-98ab-ec0848534ff0" Sep 10 03:48:10.950596 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:10.950577 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htpt9" Sep 10 03:48:10.950723 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:10.950701 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htpt9" podUID="fca9ba6d-4934-47e4-8be5-4296c7a6c530" Sep 10 03:48:10.950798 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:10.950785 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z94t" Sep 10 03:48:10.950914 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:10.950896 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z94t" podUID="72f457e8-c3eb-4b62-81e6-10cdd0f78d63" Sep 10 03:48:10.969689 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:10.968723 2581 generic.go:356] "Generic (PLEG): container finished" podID="6524cf98e47d7fba2fecd528cfc30ded" containerID="24f7a448d5f663f46b421e957ef751f482d347c763fc42e1fee37dadfb242777" exitCode=0 Sep 10 03:48:10.969689 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:10.969380 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-77.ec2.internal" event={"ID":"6524cf98e47d7fba2fecd528cfc30ded","Type":"ContainerDied","Data":"24f7a448d5f663f46b421e957ef751f482d347c763fc42e1fee37dadfb242777"} Sep 10 03:48:11.984074 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:11.984035 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-77.ec2.internal" event={"ID":"6524cf98e47d7fba2fecd528cfc30ded","Type":"ContainerStarted","Data":"3b751c088d971d9764623a9ee4d814ac8f134d26f3ddd5dc0e101c9aa25715e1"} Sep 10 03:48:11.999010 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:11.998941 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-77.ec2.internal" podStartSLOduration=3.998922462 podStartE2EDuration="3.998922462s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 03:48:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:48:11.998510708 +0000 UTC m=+5.523889816" watchObservedRunningTime="2026-09-10 03:48:11.998922462 +0000 UTC m=+5.524301568" Sep 10 03:48:12.563416 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:12.563377 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/72f457e8-c3eb-4b62-81e6-10cdd0f78d63-original-pull-secret\") pod \"global-pull-secret-syncer-2z94t\" (UID: \"72f457e8-c3eb-4b62-81e6-10cdd0f78d63\") " pod="kube-system/global-pull-secret-syncer-2z94t" Sep 10 03:48:12.563584 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:12.563451 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fca9ba6d-4934-47e4-8be5-4296c7a6c530-metrics-certs\") pod \"network-metrics-daemon-htpt9\" (UID: \"fca9ba6d-4934-47e4-8be5-4296c7a6c530\") " pod="openshift-multus/network-metrics-daemon-htpt9" Sep 10 03:48:12.563657 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:12.563598 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 03:48:12.563709 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:12.563659 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fca9ba6d-4934-47e4-8be5-4296c7a6c530-metrics-certs podName:fca9ba6d-4934-47e4-8be5-4296c7a6c530 nodeName:}" failed. No retries permitted until 2026-09-10 03:48:16.563641397 +0000 UTC m=+10.089020497 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fca9ba6d-4934-47e4-8be5-4296c7a6c530-metrics-certs") pod "network-metrics-daemon-htpt9" (UID: "fca9ba6d-4934-47e4-8be5-4296c7a6c530") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 03:48:12.564079 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:12.564061 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 03:48:12.564168 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:12.564114 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72f457e8-c3eb-4b62-81e6-10cdd0f78d63-original-pull-secret podName:72f457e8-c3eb-4b62-81e6-10cdd0f78d63 nodeName:}" failed. No retries permitted until 2026-09-10 03:48:16.564099793 +0000 UTC m=+10.089478889 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/72f457e8-c3eb-4b62-81e6-10cdd0f78d63-original-pull-secret") pod "global-pull-secret-syncer-2z94t" (UID: "72f457e8-c3eb-4b62-81e6-10cdd0f78d63") : object "kube-system"/"original-pull-secret" not registered Sep 10 03:48:12.664072 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:12.663994 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc9lh\" (UniqueName: \"kubernetes.io/projected/6be9cc5c-8c3c-4985-98ab-ec0848534ff0-kube-api-access-hc9lh\") pod \"network-check-target-7r9qf\" (UID: \"6be9cc5c-8c3c-4985-98ab-ec0848534ff0\") " pod="openshift-network-diagnostics/network-check-target-7r9qf" Sep 10 03:48:12.664270 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:12.664178 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 03:48:12.664270 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:12.664208 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 03:48:12.664270 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:12.664222 2581 projected.go:196] Error preparing data for projected volume kube-api-access-hc9lh for pod openshift-network-diagnostics/network-check-target-7r9qf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 03:48:12.664428 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:12.664295 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6be9cc5c-8c3c-4985-98ab-ec0848534ff0-kube-api-access-hc9lh podName:6be9cc5c-8c3c-4985-98ab-ec0848534ff0 nodeName:}" failed. No retries permitted until 2026-09-10 03:48:16.664275348 +0000 UTC m=+10.189654446 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-hc9lh" (UniqueName: "kubernetes.io/projected/6be9cc5c-8c3c-4985-98ab-ec0848534ff0-kube-api-access-hc9lh") pod "network-check-target-7r9qf" (UID: "6be9cc5c-8c3c-4985-98ab-ec0848534ff0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 03:48:12.938158 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:12.938080 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htpt9" Sep 10 03:48:12.938158 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:12.938100 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z94t" Sep 10 03:48:12.938158 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:12.938118 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7r9qf" Sep 10 03:48:12.938382 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:12.938230 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htpt9" podUID="fca9ba6d-4934-47e4-8be5-4296c7a6c530" Sep 10 03:48:12.938651 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:12.938626 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z94t" podUID="72f457e8-c3eb-4b62-81e6-10cdd0f78d63" Sep 10 03:48:12.938737 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:12.938717 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7r9qf" podUID="6be9cc5c-8c3c-4985-98ab-ec0848534ff0" Sep 10 03:48:14.938136 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:14.938106 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htpt9" Sep 10 03:48:14.938487 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:14.938254 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htpt9" podUID="fca9ba6d-4934-47e4-8be5-4296c7a6c530" Sep 10 03:48:14.938694 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:14.938678 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z94t" Sep 10 03:48:14.938800 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:14.938782 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z94t" podUID="72f457e8-c3eb-4b62-81e6-10cdd0f78d63" Sep 10 03:48:14.939096 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:14.938937 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7r9qf" Sep 10 03:48:14.939096 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:14.939059 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7r9qf" podUID="6be9cc5c-8c3c-4985-98ab-ec0848534ff0" Sep 10 03:48:16.596416 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:16.596186 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fca9ba6d-4934-47e4-8be5-4296c7a6c530-metrics-certs\") pod \"network-metrics-daemon-htpt9\" (UID: \"fca9ba6d-4934-47e4-8be5-4296c7a6c530\") " pod="openshift-multus/network-metrics-daemon-htpt9" Sep 10 03:48:16.596857 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:16.596482 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/72f457e8-c3eb-4b62-81e6-10cdd0f78d63-original-pull-secret\") pod \"global-pull-secret-syncer-2z94t\" (UID: \"72f457e8-c3eb-4b62-81e6-10cdd0f78d63\") " pod="kube-system/global-pull-secret-syncer-2z94t" Sep 10 03:48:16.596857 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:16.596611 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 03:48:16.596857 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:16.596661 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72f457e8-c3eb-4b62-81e6-10cdd0f78d63-original-pull-secret podName:72f457e8-c3eb-4b62-81e6-10cdd0f78d63 nodeName:}" failed. No retries permitted until 2026-09-10 03:48:24.59664407 +0000 UTC m=+18.122023154 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/72f457e8-c3eb-4b62-81e6-10cdd0f78d63-original-pull-secret") pod "global-pull-secret-syncer-2z94t" (UID: "72f457e8-c3eb-4b62-81e6-10cdd0f78d63") : object "kube-system"/"original-pull-secret" not registered Sep 10 03:48:16.597078 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:16.597042 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 03:48:16.597141 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:16.597083 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fca9ba6d-4934-47e4-8be5-4296c7a6c530-metrics-certs podName:fca9ba6d-4934-47e4-8be5-4296c7a6c530 nodeName:}" failed. No retries permitted until 2026-09-10 03:48:24.597071778 +0000 UTC m=+18.122450861 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fca9ba6d-4934-47e4-8be5-4296c7a6c530-metrics-certs") pod "network-metrics-daemon-htpt9" (UID: "fca9ba6d-4934-47e4-8be5-4296c7a6c530") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 03:48:16.697035 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:16.696999 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc9lh\" (UniqueName: \"kubernetes.io/projected/6be9cc5c-8c3c-4985-98ab-ec0848534ff0-kube-api-access-hc9lh\") pod \"network-check-target-7r9qf\" (UID: \"6be9cc5c-8c3c-4985-98ab-ec0848534ff0\") " pod="openshift-network-diagnostics/network-check-target-7r9qf" Sep 10 03:48:16.697234 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:16.697202 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 03:48:16.697234 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:16.697225 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 03:48:16.697340 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:16.697238 2581 projected.go:196] Error preparing data for projected volume kube-api-access-hc9lh for pod openshift-network-diagnostics/network-check-target-7r9qf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 03:48:16.697340 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:16.697302 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6be9cc5c-8c3c-4985-98ab-ec0848534ff0-kube-api-access-hc9lh podName:6be9cc5c-8c3c-4985-98ab-ec0848534ff0 nodeName:}" failed. No retries permitted until 2026-09-10 03:48:24.697282464 +0000 UTC m=+18.222661561 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-hc9lh" (UniqueName: "kubernetes.io/projected/6be9cc5c-8c3c-4985-98ab-ec0848534ff0-kube-api-access-hc9lh") pod "network-check-target-7r9qf" (UID: "6be9cc5c-8c3c-4985-98ab-ec0848534ff0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 03:48:16.937899 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:16.937808 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htpt9" Sep 10 03:48:16.938067 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:16.937931 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htpt9" podUID="fca9ba6d-4934-47e4-8be5-4296c7a6c530" Sep 10 03:48:16.938326 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:16.938306 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7r9qf" Sep 10 03:48:16.938423 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:16.938401 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7r9qf" podUID="6be9cc5c-8c3c-4985-98ab-ec0848534ff0" Sep 10 03:48:16.938484 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:16.938461 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z94t" Sep 10 03:48:16.938545 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:16.938528 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z94t" podUID="72f457e8-c3eb-4b62-81e6-10cdd0f78d63" Sep 10 03:48:18.936305 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:18.936274 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z94t" Sep 10 03:48:18.936683 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:18.936398 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z94t" podUID="72f457e8-c3eb-4b62-81e6-10cdd0f78d63" Sep 10 03:48:18.936683 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:18.936274 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htpt9" Sep 10 03:48:18.936754 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:18.936681 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htpt9" podUID="fca9ba6d-4934-47e4-8be5-4296c7a6c530" Sep 10 03:48:18.936847 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:18.936832 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7r9qf" Sep 10 03:48:18.936945 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:18.936925 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7r9qf" podUID="6be9cc5c-8c3c-4985-98ab-ec0848534ff0" Sep 10 03:48:20.936389 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:20.936349 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htpt9" Sep 10 03:48:20.936849 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:20.936349 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7r9qf" Sep 10 03:48:20.936849 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:20.936500 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htpt9" podUID="fca9ba6d-4934-47e4-8be5-4296c7a6c530" Sep 10 03:48:20.936849 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:20.936350 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z94t" Sep 10 03:48:20.936849 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:20.936588 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7r9qf" podUID="6be9cc5c-8c3c-4985-98ab-ec0848534ff0" Sep 10 03:48:20.936849 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:20.936652 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z94t" podUID="72f457e8-c3eb-4b62-81e6-10cdd0f78d63" Sep 10 03:48:22.936914 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:22.936878 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z94t" Sep 10 03:48:22.936914 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:22.936898 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htpt9" Sep 10 03:48:22.937438 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:22.937029 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z94t" podUID="72f457e8-c3eb-4b62-81e6-10cdd0f78d63" Sep 10 03:48:22.937438 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:22.937080 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7r9qf" Sep 10 03:48:22.937438 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:22.937181 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htpt9" podUID="fca9ba6d-4934-47e4-8be5-4296c7a6c530" Sep 10 03:48:22.937438 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:22.937245 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7r9qf" podUID="6be9cc5c-8c3c-4985-98ab-ec0848534ff0" Sep 10 03:48:24.657411 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:24.657372 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/72f457e8-c3eb-4b62-81e6-10cdd0f78d63-original-pull-secret\") pod \"global-pull-secret-syncer-2z94t\" (UID: \"72f457e8-c3eb-4b62-81e6-10cdd0f78d63\") " pod="kube-system/global-pull-secret-syncer-2z94t" Sep 10 03:48:24.657954 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:24.657438 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fca9ba6d-4934-47e4-8be5-4296c7a6c530-metrics-certs\") pod \"network-metrics-daemon-htpt9\" (UID: \"fca9ba6d-4934-47e4-8be5-4296c7a6c530\") " pod="openshift-multus/network-metrics-daemon-htpt9" Sep 10 03:48:24.657954 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:24.657550 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 03:48:24.657954 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:24.657570 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 03:48:24.657954 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:24.657632 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72f457e8-c3eb-4b62-81e6-10cdd0f78d63-original-pull-secret podName:72f457e8-c3eb-4b62-81e6-10cdd0f78d63 nodeName:}" failed. No retries permitted until 2026-09-10 03:48:40.657611726 +0000 UTC m=+34.182990824 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/72f457e8-c3eb-4b62-81e6-10cdd0f78d63-original-pull-secret") pod "global-pull-secret-syncer-2z94t" (UID: "72f457e8-c3eb-4b62-81e6-10cdd0f78d63") : object "kube-system"/"original-pull-secret" not registered Sep 10 03:48:24.657954 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:24.657655 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fca9ba6d-4934-47e4-8be5-4296c7a6c530-metrics-certs podName:fca9ba6d-4934-47e4-8be5-4296c7a6c530 nodeName:}" failed. No retries permitted until 2026-09-10 03:48:40.657643318 +0000 UTC m=+34.183022404 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fca9ba6d-4934-47e4-8be5-4296c7a6c530-metrics-certs") pod "network-metrics-daemon-htpt9" (UID: "fca9ba6d-4934-47e4-8be5-4296c7a6c530") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 03:48:24.758532 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:24.758499 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc9lh\" (UniqueName: \"kubernetes.io/projected/6be9cc5c-8c3c-4985-98ab-ec0848534ff0-kube-api-access-hc9lh\") pod \"network-check-target-7r9qf\" (UID: \"6be9cc5c-8c3c-4985-98ab-ec0848534ff0\") " pod="openshift-network-diagnostics/network-check-target-7r9qf" Sep 10 03:48:24.758690 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:24.758639 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 03:48:24.758690 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:24.758657 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 03:48:24.758690 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:24.758668 2581 projected.go:196] Error preparing data for projected volume kube-api-access-hc9lh for pod openshift-network-diagnostics/network-check-target-7r9qf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 03:48:24.758855 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:24.758729 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6be9cc5c-8c3c-4985-98ab-ec0848534ff0-kube-api-access-hc9lh podName:6be9cc5c-8c3c-4985-98ab-ec0848534ff0 nodeName:}" failed. No retries permitted until 2026-09-10 03:48:40.758712683 +0000 UTC m=+34.284091783 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-hc9lh" (UniqueName: "kubernetes.io/projected/6be9cc5c-8c3c-4985-98ab-ec0848534ff0-kube-api-access-hc9lh") pod "network-check-target-7r9qf" (UID: "6be9cc5c-8c3c-4985-98ab-ec0848534ff0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 03:48:24.936840 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:24.936758 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htpt9" Sep 10 03:48:24.937000 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:24.936758 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z94t" Sep 10 03:48:24.937000 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:24.936874 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7r9qf" Sep 10 03:48:24.937000 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:24.936876 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htpt9" podUID="fca9ba6d-4934-47e4-8be5-4296c7a6c530" Sep 10 03:48:24.937000 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:24.936940 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z94t" podUID="72f457e8-c3eb-4b62-81e6-10cdd0f78d63" Sep 10 03:48:24.937213 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:24.937028 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7r9qf" podUID="6be9cc5c-8c3c-4985-98ab-ec0848534ff0" Sep 10 03:48:26.937876 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:26.937844 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htpt9" Sep 10 03:48:26.938291 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:26.937919 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z94t" Sep 10 03:48:26.938291 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:26.938014 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7r9qf" Sep 10 03:48:26.938291 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:26.938040 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htpt9" podUID="fca9ba6d-4934-47e4-8be5-4296c7a6c530" Sep 10 03:48:26.938291 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:26.938093 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7r9qf" podUID="6be9cc5c-8c3c-4985-98ab-ec0848534ff0" Sep 10 03:48:26.938291 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:26.938138 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z94t" podUID="72f457e8-c3eb-4b62-81e6-10cdd0f78d63" Sep 10 03:48:28.022463 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:28.022433 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-sjkng" event={"ID":"271dd21f-0eed-4858-810a-687acbea64ec","Type":"ContainerStarted","Data":"acf232d5b46a982c86b3bca5b772f6a3feb1e11f9e855ef71b29ef3f84fbb7f0"} Sep 10 03:48:28.023959 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:28.023935 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-r58nd" event={"ID":"b4c0f4fb-e2d6-4ed5-969e-ec1ff4aaff92","Type":"ContainerStarted","Data":"a052bb270b01b6fcca3f7e2dd744b28041d38a97110a98811feb02fcac8e6df3"} Sep 10 03:48:28.025792 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:28.025762 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rrqkg" event={"ID":"f9d9771e-7f4b-4fd5-ac4a-f44b44379f17","Type":"ContainerStarted","Data":"6831fcd7ebb060b20723b0ec4de2a26ab1fd672313123a16781c33981c78b369"} Sep 10 03:48:28.027488 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:28.027464 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6tbmr" event={"ID":"24e969cd-fc73-4628-bcd8-0abb930691d8","Type":"ContainerStarted","Data":"a3b22b370ce20e06206abd5947f1c911c51449ec1b0b12c0e51c10dfc62bdd5a"} Sep 10 03:48:28.036228 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:28.036178 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-sjkng" podStartSLOduration=4.101713543 podStartE2EDuration="22.036161463s" totalImagesPullingTime="17.93444792s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:48:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:48:28.035605078 +0000 UTC m=+21.560984184" watchObservedRunningTime="2026-09-10 03:48:28.036161463 +0000 UTC m=+21.561540568" Sep 10 03:48:28.050185 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:28.050139 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-r58nd" podStartSLOduration=3.04994112 podStartE2EDuration="21.050122055s" totalImagesPullingTime="18.000180935s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 03:48:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:48:28.049702946 +0000 UTC m=+21.575082054" watchObservedRunningTime="2026-09-10 03:48:28.050122055 +0000 UTC m=+21.575501161" Sep 10 03:48:28.717340 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:28.717106 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-sjkng" Sep 10 03:48:28.717647 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:28.717632 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-sjkng" Sep 10 03:48:28.937093 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:28.937059 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htpt9" Sep 10 03:48:28.937300 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:28.937059 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z94t" Sep 10 03:48:28.937300 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:28.937200 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htpt9" podUID="fca9ba6d-4934-47e4-8be5-4296c7a6c530" Sep 10 03:48:28.937300 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:28.937059 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7r9qf" Sep 10 03:48:28.937300 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:28.937246 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z94t" podUID="72f457e8-c3eb-4b62-81e6-10cdd0f78d63" Sep 10 03:48:28.937502 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:28.937315 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7r9qf" podUID="6be9cc5c-8c3c-4985-98ab-ec0848534ff0" Sep 10 03:48:29.032254 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:29.031610 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7lpdd" event={"ID":"29111c2f-bef9-4076-9014-7f041915d6df","Type":"ContainerStarted","Data":"2c56f4983e6930d9d5a2e8b36850926e05c08a10da0591fc12e680e04ab2480f"} Sep 10 03:48:29.035422 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:29.035398 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" event={"ID":"f0b85b12-5116-477e-86d0-2946bc920ded","Type":"ContainerStarted","Data":"fbb02f250c1f8179cbf3f74d155f4c79c32ca84a969de507409dc4118bfae64d"} Sep 10 03:48:29.035530 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:29.035479 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" event={"ID":"f0b85b12-5116-477e-86d0-2946bc920ded","Type":"ContainerStarted","Data":"8f9e56e86a8bd0079a7a30731c2f278e351990083ea685796b955585d14a9aad"} Sep 10 03:48:29.035580 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:29.035534 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" event={"ID":"f0b85b12-5116-477e-86d0-2946bc920ded","Type":"ContainerStarted","Data":"fbf323c405f63216dca3ecf507155a0d8213bba860d46c15823028838ee39749"} Sep 10 03:48:29.035621 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:29.035581 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" event={"ID":"f0b85b12-5116-477e-86d0-2946bc920ded","Type":"ContainerStarted","Data":"9ed11c8ec4ec78539ea4aed55bffe358852c264a850c6098f587519e263b3dbd"} Sep 10 03:48:29.035670 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:29.035620 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" event={"ID":"f0b85b12-5116-477e-86d0-2946bc920ded","Type":"ContainerStarted","Data":"c4fa0fe8a54889e30b926e8da1c376c97b9d9a8e5629458eb3876bec9f8c59c8"} Sep 10 03:48:29.035670 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:29.035666 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" event={"ID":"f0b85b12-5116-477e-86d0-2946bc920ded","Type":"ContainerStarted","Data":"7456ac4f8152bed6a721561a11866dd4489cf25353e2191d16aa7bcb56e020fe"} Sep 10 03:48:29.036923 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:29.036899 2581 generic.go:356] "Generic (PLEG): container finished" podID="24e969cd-fc73-4628-bcd8-0abb930691d8" containerID="a3b22b370ce20e06206abd5947f1c911c51449ec1b0b12c0e51c10dfc62bdd5a" exitCode=0 Sep 10 03:48:29.037039 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:29.036998 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6tbmr" event={"ID":"24e969cd-fc73-4628-bcd8-0abb930691d8","Type":"ContainerDied","Data":"a3b22b370ce20e06206abd5947f1c911c51449ec1b0b12c0e51c10dfc62bdd5a"} Sep 10 03:48:29.039097 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:29.039070 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" event={"ID":"289045b6-dd39-4301-ac33-3f4fe69ad152","Type":"ContainerStarted","Data":"c5399a4134abcf20a7c7ed6af8181fd81d4b05e00f4fb054b438af60f1249af4"} Sep 10 03:48:29.042035 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:29.042010 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wnlbc" event={"ID":"1d9fdaf3-79da-4413-8a44-346fe29074be","Type":"ContainerStarted","Data":"59ac630470151f7f3e7f18770f0297d86cccfa27fabda99705a5a13f5dd82d80"} Sep 10 03:48:29.042267 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:29.042246 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-sjkng" Sep 10 03:48:29.042591 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:29.042564 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-sjkng" Sep 10 03:48:29.045709 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:29.045668 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-7lpdd" podStartSLOduration=4.042639271 podStartE2EDuration="22.045656202s" totalImagesPullingTime="18.003016931s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 03:48:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:48:29.044523456 +0000 UTC m=+22.569902560" watchObservedRunningTime="2026-09-10 03:48:29.045656202 +0000 UTC m=+22.571035307" Sep 10 03:48:29.060219 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:29.060179 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-wnlbc" podStartSLOduration=4.635966723 podStartE2EDuration="23.060167788s" totalImagesPullingTime="18.424201065s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 03:48:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:48:29.060115892 +0000 UTC m=+22.585495000" watchObservedRunningTime="2026-09-10 03:48:29.060167788 +0000 UTC m=+22.585546892" Sep 10 03:48:29.106459 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:29.106412 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-kkrg5" podStartSLOduration=4.096125202 podStartE2EDuration="22.106394647s" totalImagesPullingTime="18.010269445s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 03:48:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:48:29.106140155 +0000 UTC m=+22.631519260" watchObservedRunningTime="2026-09-10 03:48:29.106394647 +0000 UTC m=+22.631773753" Sep 10 03:48:29.109454 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:29.109435 2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 10 03:48:29.900535 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:29.900416 2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T03:48:29.109450397Z","UUID":"5eafc879-7b64-4e65-8158-479ab166a41e","Handler":null,"Name":"","Endpoint":""} Sep 10 03:48:29.902456 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:29.902255 2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 10 03:48:29.902584 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:29.902470 2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 10 03:48:30.045535 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:30.045507 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rrqkg" event={"ID":"f9d9771e-7f4b-4fd5-ac4a-f44b44379f17","Type":"ContainerStarted","Data":"3af6ef29453bb768ba183f6403ee919dce839f4ed28bfa52cb70147a06449061"} Sep 10 03:48:30.052100 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:30.052072 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tnrwv" event={"ID":"336788f7-971b-4071-80e6-afaa13696a1d","Type":"ContainerStarted","Data":"ecb5fe66c6f296ec25d7bd1e07c1c376a7480d0513b5ece05f7629d58b784117"} Sep 10 03:48:30.937136 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:30.937105 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z94t" Sep 10 03:48:30.937331 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:30.937105 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htpt9" Sep 10 03:48:30.937331 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:30.937234 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z94t" podUID="72f457e8-c3eb-4b62-81e6-10cdd0f78d63" Sep 10 03:48:30.937331 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:30.937117 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7r9qf" Sep 10 03:48:30.937479 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:30.937342 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htpt9" podUID="fca9ba6d-4934-47e4-8be5-4296c7a6c530" Sep 10 03:48:30.937479 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:30.937424 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7r9qf" podUID="6be9cc5c-8c3c-4985-98ab-ec0848534ff0" Sep 10 03:48:31.058190 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:31.058158 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" event={"ID":"f0b85b12-5116-477e-86d0-2946bc920ded","Type":"ContainerStarted","Data":"db1373ed29e344d8ba8adc951bc69514ae4b852d4a288a93a7ecad4949018d8b"} Sep 10 03:48:31.060667 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:31.060638 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rrqkg" event={"ID":"f9d9771e-7f4b-4fd5-ac4a-f44b44379f17","Type":"ContainerStarted","Data":"694726dd9d921fd971c0b8b6421913bc4f3790ac9c971c1a3f6cc3f703f841ab"} Sep 10 03:48:31.079741 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:31.079700 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-tnrwv" podStartSLOduration=6.096609785 podStartE2EDuration="24.079681656s" totalImagesPullingTime="17.983071871s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 03:48:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:48:30.067518527 +0000 UTC m=+23.592897633" watchObservedRunningTime="2026-09-10 03:48:31.079681656 +0000 UTC m=+24.605060819" Sep 10 03:48:32.937344 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:32.937258 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htpt9" Sep 10 03:48:32.937344 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:32.937310 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7r9qf" Sep 10 03:48:32.937800 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:32.937257 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z94t" Sep 10 03:48:32.937800 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:32.937393 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htpt9" podUID="fca9ba6d-4934-47e4-8be5-4296c7a6c530" Sep 10 03:48:32.937800 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:32.937501 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7r9qf" podUID="6be9cc5c-8c3c-4985-98ab-ec0848534ff0" Sep 10 03:48:32.937800 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:32.937595 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z94t" podUID="72f457e8-c3eb-4b62-81e6-10cdd0f78d63" Sep 10 03:48:34.071202 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:34.070983 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" event={"ID":"f0b85b12-5116-477e-86d0-2946bc920ded","Type":"ContainerStarted","Data":"efba7c8cfc270bfe42bbec81a5037dec74525aad4f7f3ae60e08df791d84dee4"} Sep 10 03:48:34.071644 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:34.071292 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:34.072988 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:34.072936 2581 generic.go:356] "Generic (PLEG): container finished" podID="24e969cd-fc73-4628-bcd8-0abb930691d8" containerID="c5ab1888d45442b31fa11592e830896368a6d746a0e3af90b59e2d87b94fdbed" exitCode=0 Sep 10 03:48:34.073099 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:34.072999 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6tbmr" event={"ID":"24e969cd-fc73-4628-bcd8-0abb930691d8","Type":"ContainerDied","Data":"c5ab1888d45442b31fa11592e830896368a6d746a0e3af90b59e2d87b94fdbed"} Sep 10 03:48:34.088492 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:34.088465 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:34.096110 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:34.096066 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rrqkg" podStartSLOduration=7.230690314 podStartE2EDuration="27.096050066s" totalImagesPullingTime="19.865359752s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 03:48:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:48:31.080221709 +0000 UTC m=+24.605600815" watchObservedRunningTime="2026-09-10 03:48:34.096050066 +0000 UTC m=+27.621429169" Sep 10 03:48:34.096616 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:34.096591 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" podStartSLOduration=9.074919301 podStartE2EDuration="27.096580253s" totalImagesPullingTime="18.021660952s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 03:48:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:48:34.095925357 +0000 UTC m=+27.621304462" watchObservedRunningTime="2026-09-10 03:48:34.096580253 +0000 UTC m=+27.621959358" Sep 10 03:48:34.936837 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:34.936799 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z94t" Sep 10 03:48:34.937245 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:34.936805 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7r9qf" Sep 10 03:48:34.937245 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:34.936901 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z94t" podUID="72f457e8-c3eb-4b62-81e6-10cdd0f78d63" Sep 10 03:48:34.937245 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:34.936915 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htpt9" Sep 10 03:48:34.937245 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:34.937006 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7r9qf" podUID="6be9cc5c-8c3c-4985-98ab-ec0848534ff0" Sep 10 03:48:34.937245 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:34.937101 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htpt9" podUID="fca9ba6d-4934-47e4-8be5-4296c7a6c530" Sep 10 03:48:35.075860 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:35.075822 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:35.076528 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:35.075893 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:35.090615 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:35.090593 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:48:35.397004 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:35.396961 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2z94t"] Sep 10 03:48:35.397137 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:35.397124 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z94t" Sep 10 03:48:35.397237 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:35.397218 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z94t" podUID="72f457e8-c3eb-4b62-81e6-10cdd0f78d63" Sep 10 03:48:35.397581 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:35.397562 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7r9qf"] Sep 10 03:48:35.397665 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:35.397648 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7r9qf" Sep 10 03:48:35.397745 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:35.397722 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7r9qf" podUID="6be9cc5c-8c3c-4985-98ab-ec0848534ff0" Sep 10 03:48:35.399427 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:35.399401 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-htpt9"] Sep 10 03:48:35.399510 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:35.399485 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htpt9" Sep 10 03:48:35.399572 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:35.399558 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htpt9" podUID="fca9ba6d-4934-47e4-8be5-4296c7a6c530" Sep 10 03:48:36.079819 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:36.079787 2581 generic.go:356] "Generic (PLEG): container finished" podID="24e969cd-fc73-4628-bcd8-0abb930691d8" containerID="495377e5224d57717a416e4e2122db7635317f4db88fe2bd89b9591f5ff1c9b6" exitCode=0 Sep 10 03:48:36.080537 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:36.079877 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6tbmr" event={"ID":"24e969cd-fc73-4628-bcd8-0abb930691d8","Type":"ContainerDied","Data":"495377e5224d57717a416e4e2122db7635317f4db88fe2bd89b9591f5ff1c9b6"} Sep 10 03:48:36.937425 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:36.937271 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htpt9" Sep 10 03:48:36.937572 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:36.937485 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htpt9" podUID="fca9ba6d-4934-47e4-8be5-4296c7a6c530" Sep 10 03:48:36.937572 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:36.937348 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7r9qf" Sep 10 03:48:36.937572 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:36.937540 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7r9qf" podUID="6be9cc5c-8c3c-4985-98ab-ec0848534ff0" Sep 10 03:48:36.937572 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:36.937334 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z94t" Sep 10 03:48:36.937715 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:36.937592 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z94t" podUID="72f457e8-c3eb-4b62-81e6-10cdd0f78d63" Sep 10 03:48:37.085368 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:37.085339 2581 generic.go:356] "Generic (PLEG): container finished" podID="24e969cd-fc73-4628-bcd8-0abb930691d8" containerID="0450d81eab216d83fbe28f7800e6e9ff65f818166ee9581938ed9bf64da4b792" exitCode=0 Sep 10 03:48:37.085819 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:37.085422 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6tbmr" event={"ID":"24e969cd-fc73-4628-bcd8-0abb930691d8","Type":"ContainerDied","Data":"0450d81eab216d83fbe28f7800e6e9ff65f818166ee9581938ed9bf64da4b792"} Sep 10 03:48:38.937034 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:38.937001 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z94t" Sep 10 03:48:38.937496 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:38.937001 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htpt9" Sep 10 03:48:38.937496 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:38.937133 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2z94t" podUID="72f457e8-c3eb-4b62-81e6-10cdd0f78d63" Sep 10 03:48:38.937496 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:38.937210 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-htpt9" podUID="fca9ba6d-4934-47e4-8be5-4296c7a6c530" Sep 10 03:48:38.937496 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:38.937006 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7r9qf" Sep 10 03:48:38.937496 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:38.937296 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7r9qf" podUID="6be9cc5c-8c3c-4985-98ab-ec0848534ff0" Sep 10 03:48:40.678699 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:40.678668 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/72f457e8-c3eb-4b62-81e6-10cdd0f78d63-original-pull-secret\") pod \"global-pull-secret-syncer-2z94t\" (UID: \"72f457e8-c3eb-4b62-81e6-10cdd0f78d63\") " pod="kube-system/global-pull-secret-syncer-2z94t" Sep 10 03:48:40.679102 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:40.678720 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fca9ba6d-4934-47e4-8be5-4296c7a6c530-metrics-certs\") pod \"network-metrics-daemon-htpt9\" (UID: \"fca9ba6d-4934-47e4-8be5-4296c7a6c530\") " pod="openshift-multus/network-metrics-daemon-htpt9" Sep 10 03:48:40.679102 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:40.678828 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 03:48:40.679102 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:40.678832 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 03:48:40.679102 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:40.678894 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fca9ba6d-4934-47e4-8be5-4296c7a6c530-metrics-certs podName:fca9ba6d-4934-47e4-8be5-4296c7a6c530 nodeName:}" failed. No retries permitted until 2026-09-10 03:49:12.678879236 +0000 UTC m=+66.204258323 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fca9ba6d-4934-47e4-8be5-4296c7a6c530-metrics-certs") pod "network-metrics-daemon-htpt9" (UID: "fca9ba6d-4934-47e4-8be5-4296c7a6c530") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 03:48:40.679102 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:40.678908 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72f457e8-c3eb-4b62-81e6-10cdd0f78d63-original-pull-secret podName:72f457e8-c3eb-4b62-81e6-10cdd0f78d63 nodeName:}" failed. No retries permitted until 2026-09-10 03:49:12.678902395 +0000 UTC m=+66.204281480 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/72f457e8-c3eb-4b62-81e6-10cdd0f78d63-original-pull-secret") pod "global-pull-secret-syncer-2z94t" (UID: "72f457e8-c3eb-4b62-81e6-10cdd0f78d63") : object "kube-system"/"original-pull-secret" not registered Sep 10 03:48:40.780152 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:40.779958 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc9lh\" (UniqueName: \"kubernetes.io/projected/6be9cc5c-8c3c-4985-98ab-ec0848534ff0-kube-api-access-hc9lh\") pod \"network-check-target-7r9qf\" (UID: \"6be9cc5c-8c3c-4985-98ab-ec0848534ff0\") " pod="openshift-network-diagnostics/network-check-target-7r9qf" Sep 10 03:48:40.780321 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:40.780166 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 03:48:40.780321 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:40.780185 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 03:48:40.780321 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:40.780196 2581 projected.go:196] Error preparing data for projected volume kube-api-access-hc9lh for pod openshift-network-diagnostics/network-check-target-7r9qf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 03:48:40.780321 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:40.780255 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6be9cc5c-8c3c-4985-98ab-ec0848534ff0-kube-api-access-hc9lh podName:6be9cc5c-8c3c-4985-98ab-ec0848534ff0 nodeName:}" failed. No retries permitted until 2026-09-10 03:49:12.780238899 +0000 UTC m=+66.305617995 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-hc9lh" (UniqueName: "kubernetes.io/projected/6be9cc5c-8c3c-4985-98ab-ec0848534ff0-kube-api-access-hc9lh") pod "network-check-target-7r9qf" (UID: "6be9cc5c-8c3c-4985-98ab-ec0848534ff0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 03:48:40.798218 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:40.798192 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-77.ec2.internal" event="NodeReady" Sep 10 03:48:40.798356 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:40.798330 2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 10 03:48:40.846076 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:40.846033 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-kzds7"] Sep 10 03:48:40.867778 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:40.867744 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-cmdjp"] Sep 10 03:48:40.867938 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:40.867870 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kzds7" Sep 10 03:48:40.870714 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:40.870686 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 10 03:48:40.871370 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:40.871349 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bt2fn\"" Sep 10 03:48:40.871467 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:40.871451 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 10 03:48:40.879121 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:40.879101 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-kzds7"] Sep 10 03:48:40.879237 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:40.879140 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-cmdjp"] Sep 10 03:48:40.879237 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:40.879227 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-cmdjp" Sep 10 03:48:40.881895 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:40.881874 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 10 03:48:40.882021 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:40.881879 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 10 03:48:40.882193 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:40.882173 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 10 03:48:40.882310 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:40.882173 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-cprh9\"" Sep 10 03:48:40.937393 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:40.937362 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z94t" Sep 10 03:48:40.937587 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:40.937363 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7r9qf" Sep 10 03:48:40.937587 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:40.937483 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htpt9" Sep 10 03:48:40.939927 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:40.939908 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 10 03:48:40.939927 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:40.939917 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 10 03:48:40.940210 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:40.940190 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7r79s\"" Sep 10 03:48:40.940293 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:40.940209 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 10 03:48:40.940293 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:40.940215 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 10 03:48:40.940405 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:40.940296 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2jbt8\"" Sep 10 03:48:40.981207 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:40.981181 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgcg9\" (UniqueName: \"kubernetes.io/projected/59ba01f7-c3bf-4ccb-8a19-f69bbda08d5f-kube-api-access-fgcg9\") pod \"ingress-canary-cmdjp\" (UID: \"59ba01f7-c3bf-4ccb-8a19-f69bbda08d5f\") " pod="openshift-ingress-canary/ingress-canary-cmdjp" Sep 10 03:48:40.981337 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:40.981225 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/184e14a2-418b-41da-8635-23ea4c965dbc-metrics-tls\") pod \"dns-default-kzds7\" (UID: \"184e14a2-418b-41da-8635-23ea4c965dbc\") " pod="openshift-dns/dns-default-kzds7" Sep 10 03:48:40.981337 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:40.981245 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/184e14a2-418b-41da-8635-23ea4c965dbc-tmp-dir\") pod \"dns-default-kzds7\" (UID: \"184e14a2-418b-41da-8635-23ea4c965dbc\") " pod="openshift-dns/dns-default-kzds7" Sep 10 03:48:40.981337 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:40.981269 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lcxl\" (UniqueName: \"kubernetes.io/projected/184e14a2-418b-41da-8635-23ea4c965dbc-kube-api-access-8lcxl\") pod \"dns-default-kzds7\" (UID: \"184e14a2-418b-41da-8635-23ea4c965dbc\") " pod="openshift-dns/dns-default-kzds7" Sep 10 03:48:40.981490 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:40.981346 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/184e14a2-418b-41da-8635-23ea4c965dbc-config-volume\") pod \"dns-default-kzds7\" (UID: \"184e14a2-418b-41da-8635-23ea4c965dbc\") " pod="openshift-dns/dns-default-kzds7" Sep 10 03:48:40.981490 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:40.981367 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/59ba01f7-c3bf-4ccb-8a19-f69bbda08d5f-cert\") pod \"ingress-canary-cmdjp\" (UID: \"59ba01f7-c3bf-4ccb-8a19-f69bbda08d5f\") " pod="openshift-ingress-canary/ingress-canary-cmdjp" Sep 10 03:48:41.082628 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:41.082588 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/184e14a2-418b-41da-8635-23ea4c965dbc-metrics-tls\") pod \"dns-default-kzds7\" (UID: \"184e14a2-418b-41da-8635-23ea4c965dbc\") " pod="openshift-dns/dns-default-kzds7" Sep 10 03:48:41.082628 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:41.082629 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/184e14a2-418b-41da-8635-23ea4c965dbc-tmp-dir\") pod \"dns-default-kzds7\" (UID: \"184e14a2-418b-41da-8635-23ea4c965dbc\") " pod="openshift-dns/dns-default-kzds7" Sep 10 03:48:41.082894 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:41.082650 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lcxl\" (UniqueName: \"kubernetes.io/projected/184e14a2-418b-41da-8635-23ea4c965dbc-kube-api-access-8lcxl\") pod \"dns-default-kzds7\" (UID: \"184e14a2-418b-41da-8635-23ea4c965dbc\") " pod="openshift-dns/dns-default-kzds7" Sep 10 03:48:41.082894 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:41.082747 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 03:48:41.082894 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:41.082810 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/184e14a2-418b-41da-8635-23ea4c965dbc-metrics-tls podName:184e14a2-418b-41da-8635-23ea4c965dbc nodeName:}" failed. No retries permitted until 2026-09-10 03:48:41.582792019 +0000 UTC m=+35.108171119 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/184e14a2-418b-41da-8635-23ea4c965dbc-metrics-tls") pod "dns-default-kzds7" (UID: "184e14a2-418b-41da-8635-23ea4c965dbc") : secret "dns-default-metrics-tls" not found Sep 10 03:48:41.083025 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:41.082942 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/184e14a2-418b-41da-8635-23ea4c965dbc-tmp-dir\") pod \"dns-default-kzds7\" (UID: \"184e14a2-418b-41da-8635-23ea4c965dbc\") " pod="openshift-dns/dns-default-kzds7" Sep 10 03:48:41.083025 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:41.083004 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/184e14a2-418b-41da-8635-23ea4c965dbc-config-volume\") pod \"dns-default-kzds7\" (UID: \"184e14a2-418b-41da-8635-23ea4c965dbc\") " pod="openshift-dns/dns-default-kzds7" Sep 10 03:48:41.083086 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:41.083036 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/59ba01f7-c3bf-4ccb-8a19-f69bbda08d5f-cert\") pod \"ingress-canary-cmdjp\" (UID: \"59ba01f7-c3bf-4ccb-8a19-f69bbda08d5f\") " pod="openshift-ingress-canary/ingress-canary-cmdjp" Sep 10 03:48:41.083086 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:41.083066 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgcg9\" (UniqueName: \"kubernetes.io/projected/59ba01f7-c3bf-4ccb-8a19-f69bbda08d5f-kube-api-access-fgcg9\") pod \"ingress-canary-cmdjp\" (UID: \"59ba01f7-c3bf-4ccb-8a19-f69bbda08d5f\") " pod="openshift-ingress-canary/ingress-canary-cmdjp" Sep 10 03:48:41.083461 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:41.083445 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/184e14a2-418b-41da-8635-23ea4c965dbc-config-volume\") pod \"dns-default-kzds7\" (UID: \"184e14a2-418b-41da-8635-23ea4c965dbc\") " pod="openshift-dns/dns-default-kzds7" Sep 10 03:48:41.087713 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:41.087689 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/59ba01f7-c3bf-4ccb-8a19-f69bbda08d5f-cert\") pod \"ingress-canary-cmdjp\" (UID: \"59ba01f7-c3bf-4ccb-8a19-f69bbda08d5f\") " pod="openshift-ingress-canary/ingress-canary-cmdjp" Sep 10 03:48:41.091898 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:41.091876 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lcxl\" (UniqueName: \"kubernetes.io/projected/184e14a2-418b-41da-8635-23ea4c965dbc-kube-api-access-8lcxl\") pod \"dns-default-kzds7\" (UID: \"184e14a2-418b-41da-8635-23ea4c965dbc\") " pod="openshift-dns/dns-default-kzds7" Sep 10 03:48:41.093223 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:41.093202 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgcg9\" (UniqueName: \"kubernetes.io/projected/59ba01f7-c3bf-4ccb-8a19-f69bbda08d5f-kube-api-access-fgcg9\") pod \"ingress-canary-cmdjp\" (UID: \"59ba01f7-c3bf-4ccb-8a19-f69bbda08d5f\") " pod="openshift-ingress-canary/ingress-canary-cmdjp" Sep 10 03:48:41.188781 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:41.188719 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-cmdjp" Sep 10 03:48:41.487637 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:41.487411 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-cmdjp"] Sep 10 03:48:41.492895 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:48:41.492870 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59ba01f7_c3bf_4ccb_8a19_f69bbda08d5f.slice/crio-b3cda21b0e85a0ceb1b599775ad46caddcc5b699cb742f16659534520da68961 WatchSource:0}: Error finding container b3cda21b0e85a0ceb1b599775ad46caddcc5b699cb742f16659534520da68961: Status 404 returned error can't find the container with id b3cda21b0e85a0ceb1b599775ad46caddcc5b699cb742f16659534520da68961 Sep 10 03:48:41.586848 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:41.586807 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/184e14a2-418b-41da-8635-23ea4c965dbc-metrics-tls\") pod \"dns-default-kzds7\" (UID: \"184e14a2-418b-41da-8635-23ea4c965dbc\") " pod="openshift-dns/dns-default-kzds7" Sep 10 03:48:41.586988 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:41.586943 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 03:48:41.587033 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:41.587020 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/184e14a2-418b-41da-8635-23ea4c965dbc-metrics-tls podName:184e14a2-418b-41da-8635-23ea4c965dbc nodeName:}" failed. No retries permitted until 2026-09-10 03:48:42.58700599 +0000 UTC m=+36.112385074 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/184e14a2-418b-41da-8635-23ea4c965dbc-metrics-tls") pod "dns-default-kzds7" (UID: "184e14a2-418b-41da-8635-23ea4c965dbc") : secret "dns-default-metrics-tls" not found Sep 10 03:48:42.099598 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:42.099568 2581 generic.go:356] "Generic (PLEG): container finished" podID="24e969cd-fc73-4628-bcd8-0abb930691d8" containerID="7da3f3197c787a9e2065c7e72d556003d4e7dae92982b92e5333efd8c3731d9c" exitCode=0 Sep 10 03:48:42.100180 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:42.099637 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6tbmr" event={"ID":"24e969cd-fc73-4628-bcd8-0abb930691d8","Type":"ContainerDied","Data":"7da3f3197c787a9e2065c7e72d556003d4e7dae92982b92e5333efd8c3731d9c"} Sep 10 03:48:42.100889 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:42.100866 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-cmdjp" event={"ID":"59ba01f7-c3bf-4ccb-8a19-f69bbda08d5f","Type":"ContainerStarted","Data":"b3cda21b0e85a0ceb1b599775ad46caddcc5b699cb742f16659534520da68961"} Sep 10 03:48:42.596061 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:42.596025 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/184e14a2-418b-41da-8635-23ea4c965dbc-metrics-tls\") pod \"dns-default-kzds7\" (UID: \"184e14a2-418b-41da-8635-23ea4c965dbc\") " pod="openshift-dns/dns-default-kzds7" Sep 10 03:48:42.596230 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:42.596160 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 03:48:42.596289 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:42.596231 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/184e14a2-418b-41da-8635-23ea4c965dbc-metrics-tls podName:184e14a2-418b-41da-8635-23ea4c965dbc nodeName:}" failed. No retries permitted until 2026-09-10 03:48:44.596211413 +0000 UTC m=+38.121590503 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/184e14a2-418b-41da-8635-23ea4c965dbc-metrics-tls") pod "dns-default-kzds7" (UID: "184e14a2-418b-41da-8635-23ea4c965dbc") : secret "dns-default-metrics-tls" not found Sep 10 03:48:43.109909 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:43.109870 2581 generic.go:356] "Generic (PLEG): container finished" podID="24e969cd-fc73-4628-bcd8-0abb930691d8" containerID="074b18770c36bb7585e9ab63437f967c7c1c15388b3f32ac17b120f603633f9d" exitCode=0 Sep 10 03:48:43.110370 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:43.109916 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6tbmr" event={"ID":"24e969cd-fc73-4628-bcd8-0abb930691d8","Type":"ContainerDied","Data":"074b18770c36bb7585e9ab63437f967c7c1c15388b3f32ac17b120f603633f9d"} Sep 10 03:48:44.116523 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:44.116493 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6tbmr" event={"ID":"24e969cd-fc73-4628-bcd8-0abb930691d8","Type":"ContainerStarted","Data":"f281f369dd4bff420052125117d5d4fa88fa82e4195da56f084eaef6c6ebdb0a"} Sep 10 03:48:44.137684 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:44.137630 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-6tbmr" podStartSLOduration=8.319911307 podStartE2EDuration="37.137611243s" totalImagesPullingTime="28.817699936s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 03:48:07 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:48:44.136374285 +0000 UTC m=+37.661753387" watchObservedRunningTime="2026-09-10 03:48:44.137611243 +0000 UTC m=+37.662990349" Sep 10 03:48:44.611203 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:44.611172 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/184e14a2-418b-41da-8635-23ea4c965dbc-metrics-tls\") pod \"dns-default-kzds7\" (UID: \"184e14a2-418b-41da-8635-23ea4c965dbc\") " pod="openshift-dns/dns-default-kzds7" Sep 10 03:48:44.611352 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:44.611327 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 03:48:44.611404 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:44.611394 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/184e14a2-418b-41da-8635-23ea4c965dbc-metrics-tls podName:184e14a2-418b-41da-8635-23ea4c965dbc nodeName:}" failed. No retries permitted until 2026-09-10 03:48:48.6113775 +0000 UTC m=+42.136756584 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/184e14a2-418b-41da-8635-23ea4c965dbc-metrics-tls") pod "dns-default-kzds7" (UID: "184e14a2-418b-41da-8635-23ea4c965dbc") : secret "dns-default-metrics-tls" not found Sep 10 03:48:45.120879 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:45.120624 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-cmdjp" event={"ID":"59ba01f7-c3bf-4ccb-8a19-f69bbda08d5f","Type":"ContainerStarted","Data":"5610b83f26efebf55a0963becc5ba511505d1413ffa151d05b70d28a20bd4f76"} Sep 10 03:48:45.135826 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:45.135780 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-cmdjp" podStartSLOduration=1.871330361 podStartE2EDuration="5.13576592s" totalImagesPullingTime="3.264435559s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:48:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:48:45.134784593 +0000 UTC m=+38.660163698" watchObservedRunningTime="2026-09-10 03:48:45.13576592 +0000 UTC m=+38.661145021" Sep 10 03:48:48.641848 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:48.641803 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/184e14a2-418b-41da-8635-23ea4c965dbc-metrics-tls\") pod \"dns-default-kzds7\" (UID: \"184e14a2-418b-41da-8635-23ea4c965dbc\") " pod="openshift-dns/dns-default-kzds7" Sep 10 03:48:48.642288 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:48.641959 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 03:48:48.642288 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:48.642044 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/184e14a2-418b-41da-8635-23ea4c965dbc-metrics-tls podName:184e14a2-418b-41da-8635-23ea4c965dbc nodeName:}" failed. No retries permitted until 2026-09-10 03:48:56.642028581 +0000 UTC m=+50.167407685 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/184e14a2-418b-41da-8635-23ea4c965dbc-metrics-tls") pod "dns-default-kzds7" (UID: "184e14a2-418b-41da-8635-23ea4c965dbc") : secret "dns-default-metrics-tls" not found Sep 10 03:48:56.699712 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:48:56.699674 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/184e14a2-418b-41da-8635-23ea4c965dbc-metrics-tls\") pod \"dns-default-kzds7\" (UID: \"184e14a2-418b-41da-8635-23ea4c965dbc\") " pod="openshift-dns/dns-default-kzds7" Sep 10 03:48:56.700098 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:56.699833 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 03:48:56.700098 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:48:56.699902 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/184e14a2-418b-41da-8635-23ea4c965dbc-metrics-tls podName:184e14a2-418b-41da-8635-23ea4c965dbc nodeName:}" failed. No retries permitted until 2026-09-10 03:49:12.699884266 +0000 UTC m=+66.225263350 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/184e14a2-418b-41da-8635-23ea4c965dbc-metrics-tls") pod "dns-default-kzds7" (UID: "184e14a2-418b-41da-8635-23ea4c965dbc") : secret "dns-default-metrics-tls" not found Sep 10 03:49:07.096328 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:07.096299 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cqmkb" Sep 10 03:49:12.711902 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:12.711856 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/184e14a2-418b-41da-8635-23ea4c965dbc-metrics-tls\") pod \"dns-default-kzds7\" (UID: \"184e14a2-418b-41da-8635-23ea4c965dbc\") " pod="openshift-dns/dns-default-kzds7" Sep 10 03:49:12.711902 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:12.711911 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/72f457e8-c3eb-4b62-81e6-10cdd0f78d63-original-pull-secret\") pod \"global-pull-secret-syncer-2z94t\" (UID: \"72f457e8-c3eb-4b62-81e6-10cdd0f78d63\") " pod="kube-system/global-pull-secret-syncer-2z94t" Sep 10 03:49:12.712366 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:49:12.712022 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 03:49:12.712366 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:12.712043 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fca9ba6d-4934-47e4-8be5-4296c7a6c530-metrics-certs\") pod \"network-metrics-daemon-htpt9\" (UID: \"fca9ba6d-4934-47e4-8be5-4296c7a6c530\") " pod="openshift-multus/network-metrics-daemon-htpt9" Sep 10 03:49:12.712366 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:49:12.712092 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/184e14a2-418b-41da-8635-23ea4c965dbc-metrics-tls podName:184e14a2-418b-41da-8635-23ea4c965dbc nodeName:}" failed. No retries permitted until 2026-09-10 03:49:44.712077659 +0000 UTC m=+98.237456745 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/184e14a2-418b-41da-8635-23ea4c965dbc-metrics-tls") pod "dns-default-kzds7" (UID: "184e14a2-418b-41da-8635-23ea4c965dbc") : secret "dns-default-metrics-tls" not found Sep 10 03:49:12.714829 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:12.714804 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 10 03:49:12.714957 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:12.714884 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 10 03:49:12.723049 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:49:12.723026 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 10 03:49:12.723150 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:49:12.723081 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fca9ba6d-4934-47e4-8be5-4296c7a6c530-metrics-certs podName:fca9ba6d-4934-47e4-8be5-4296c7a6c530 nodeName:}" failed. No retries permitted until 2026-09-10 03:50:16.723068057 +0000 UTC m=+130.248447140 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fca9ba6d-4934-47e4-8be5-4296c7a6c530-metrics-certs") pod "network-metrics-daemon-htpt9" (UID: "fca9ba6d-4934-47e4-8be5-4296c7a6c530") : secret "metrics-daemon-secret" not found Sep 10 03:49:12.726403 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:12.726384 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/72f457e8-c3eb-4b62-81e6-10cdd0f78d63-original-pull-secret\") pod \"global-pull-secret-syncer-2z94t\" (UID: \"72f457e8-c3eb-4b62-81e6-10cdd0f78d63\") " pod="kube-system/global-pull-secret-syncer-2z94t" Sep 10 03:49:12.749436 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:12.749405 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2z94t" Sep 10 03:49:12.812791 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:12.812743 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc9lh\" (UniqueName: \"kubernetes.io/projected/6be9cc5c-8c3c-4985-98ab-ec0848534ff0-kube-api-access-hc9lh\") pod \"network-check-target-7r9qf\" (UID: \"6be9cc5c-8c3c-4985-98ab-ec0848534ff0\") " pod="openshift-network-diagnostics/network-check-target-7r9qf" Sep 10 03:49:12.816049 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:12.816029 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 10 03:49:12.825434 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:12.825414 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 10 03:49:12.835625 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:12.835604 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hc9lh\" (UniqueName: \"kubernetes.io/projected/6be9cc5c-8c3c-4985-98ab-ec0848534ff0-kube-api-access-hc9lh\") pod \"network-check-target-7r9qf\" (UID: \"6be9cc5c-8c3c-4985-98ab-ec0848534ff0\") " pod="openshift-network-diagnostics/network-check-target-7r9qf" Sep 10 03:49:13.017575 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:13.017497 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2z94t"] Sep 10 03:49:13.022096 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:49:13.022069 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72f457e8_c3eb_4b62_81e6_10cdd0f78d63.slice/crio-98d76159679eda6c72a0a64dae8b64ee8f42b5dfb9f921826e07673e7646ebe5 WatchSource:0}: Error finding container 98d76159679eda6c72a0a64dae8b64ee8f42b5dfb9f921826e07673e7646ebe5: Status 404 returned error can't find the container with id 98d76159679eda6c72a0a64dae8b64ee8f42b5dfb9f921826e07673e7646ebe5 Sep 10 03:49:13.064028 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:13.064003 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2jbt8\"" Sep 10 03:49:13.072023 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:13.072011 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7r9qf" Sep 10 03:49:13.186216 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:13.186185 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2z94t" event={"ID":"72f457e8-c3eb-4b62-81e6-10cdd0f78d63","Type":"ContainerStarted","Data":"98d76159679eda6c72a0a64dae8b64ee8f42b5dfb9f921826e07673e7646ebe5"} Sep 10 03:49:13.338939 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:13.338909 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7r9qf"] Sep 10 03:49:13.342534 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:49:13.342505 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6be9cc5c_8c3c_4985_98ab_ec0848534ff0.slice/crio-7d606edf8ca0673dd6c352009c1d85f94127dfbb03fb30f3d0a00b51996e6913 WatchSource:0}: Error finding container 7d606edf8ca0673dd6c352009c1d85f94127dfbb03fb30f3d0a00b51996e6913: Status 404 returned error can't find the container with id 7d606edf8ca0673dd6c352009c1d85f94127dfbb03fb30f3d0a00b51996e6913 Sep 10 03:49:14.190842 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:14.190792 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7r9qf" event={"ID":"6be9cc5c-8c3c-4985-98ab-ec0848534ff0","Type":"ContainerStarted","Data":"7d606edf8ca0673dd6c352009c1d85f94127dfbb03fb30f3d0a00b51996e6913"} Sep 10 03:49:18.202813 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:18.202717 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2z94t" event={"ID":"72f457e8-c3eb-4b62-81e6-10cdd0f78d63","Type":"ContainerStarted","Data":"fe138a52b4291c454bffac343abdd686114d1e9270e3225555f02ffeacf9bf18"} Sep 10 03:49:18.204263 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:18.204239 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7r9qf" event={"ID":"6be9cc5c-8c3c-4985-98ab-ec0848534ff0","Type":"ContainerStarted","Data":"e5b10f9a782323a60b5e6da0c650a45452293da8c05e83571580399c8e48c469"} Sep 10 03:49:18.204380 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:18.204294 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-7r9qf" Sep 10 03:49:18.218247 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:18.218202 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-2z94t" podStartSLOduration=66.36786719 podStartE2EDuration="1m11.218188401s" totalImagesPullingTime="4.850321211s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 03:48:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:49:18.217540546 +0000 UTC m=+71.742919652" watchObservedRunningTime="2026-09-10 03:49:18.218188401 +0000 UTC m=+71.743567506" Sep 10 03:49:18.231401 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:18.231359 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-7r9qf" podStartSLOduration=66.705287891 podStartE2EDuration="1m11.231344194s" totalImagesPullingTime="4.526056303s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:48:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:49:18.230298322 +0000 UTC m=+71.755677427" watchObservedRunningTime="2026-09-10 03:49:18.231344194 +0000 UTC m=+71.756723298" Sep 10 03:49:40.838989 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:40.838862 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-69484f69f8-6tzjr"] Sep 10 03:49:40.846094 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:40.846070 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69484f69f8-6tzjr" Sep 10 03:49:40.849713 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:40.849689 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 10 03:49:40.849837 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:40.849720 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 10 03:49:40.849837 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:40.849693 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Sep 10 03:49:40.849837 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:40.849693 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 10 03:49:40.850483 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:40.850464 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-69484f69f8-6tzjr"] Sep 10 03:49:40.879208 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:40.879185 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d9676fc7b-6zjvl"] Sep 10 03:49:40.881965 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:40.881952 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d9676fc7b-6zjvl" Sep 10 03:49:40.884316 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:40.884279 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Sep 10 03:49:40.884413 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:40.884330 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Sep 10 03:49:40.884413 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:40.884352 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Sep 10 03:49:40.884413 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:40.884290 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Sep 10 03:49:40.892590 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:40.892573 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d9676fc7b-6zjvl"] Sep 10 03:49:40.901737 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:40.901719 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/005bbc1c-6e3d-4b66-b0db-4af80d7e525a-ca\") pod \"cluster-proxy-proxy-agent-5d9676fc7b-6zjvl\" (UID: \"005bbc1c-6e3d-4b66-b0db-4af80d7e525a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d9676fc7b-6zjvl" Sep 10 03:49:40.901824 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:40.901743 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/005bbc1c-6e3d-4b66-b0db-4af80d7e525a-hub\") pod \"cluster-proxy-proxy-agent-5d9676fc7b-6zjvl\" (UID: \"005bbc1c-6e3d-4b66-b0db-4af80d7e525a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d9676fc7b-6zjvl" Sep 10 03:49:40.901824 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:40.901759 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/005bbc1c-6e3d-4b66-b0db-4af80d7e525a-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5d9676fc7b-6zjvl\" (UID: \"005bbc1c-6e3d-4b66-b0db-4af80d7e525a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d9676fc7b-6zjvl" Sep 10 03:49:40.901824 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:40.901781 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/785a605a-5323-4c5d-934c-ff52f0e0043b-klusterlet-config\") pod \"klusterlet-addon-workmgr-69484f69f8-6tzjr\" (UID: \"785a605a-5323-4c5d-934c-ff52f0e0043b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69484f69f8-6tzjr" Sep 10 03:49:40.901941 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:40.901871 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5s9x\" (UniqueName: \"kubernetes.io/projected/785a605a-5323-4c5d-934c-ff52f0e0043b-kube-api-access-v5s9x\") pod \"klusterlet-addon-workmgr-69484f69f8-6tzjr\" (UID: \"785a605a-5323-4c5d-934c-ff52f0e0043b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69484f69f8-6tzjr" Sep 10 03:49:40.901941 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:40.901899 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vnh2\" (UniqueName: \"kubernetes.io/projected/005bbc1c-6e3d-4b66-b0db-4af80d7e525a-kube-api-access-5vnh2\") pod \"cluster-proxy-proxy-agent-5d9676fc7b-6zjvl\" (UID: \"005bbc1c-6e3d-4b66-b0db-4af80d7e525a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d9676fc7b-6zjvl" Sep 10 03:49:40.901941 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:40.901936 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/785a605a-5323-4c5d-934c-ff52f0e0043b-tmp\") pod \"klusterlet-addon-workmgr-69484f69f8-6tzjr\" (UID: \"785a605a-5323-4c5d-934c-ff52f0e0043b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69484f69f8-6tzjr" Sep 10 03:49:40.902069 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:40.902009 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/005bbc1c-6e3d-4b66-b0db-4af80d7e525a-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5d9676fc7b-6zjvl\" (UID: \"005bbc1c-6e3d-4b66-b0db-4af80d7e525a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d9676fc7b-6zjvl" Sep 10 03:49:40.902069 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:40.902037 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/005bbc1c-6e3d-4b66-b0db-4af80d7e525a-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5d9676fc7b-6zjvl\" (UID: \"005bbc1c-6e3d-4b66-b0db-4af80d7e525a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d9676fc7b-6zjvl" Sep 10 03:49:41.002593 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:41.002556 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5s9x\" (UniqueName: \"kubernetes.io/projected/785a605a-5323-4c5d-934c-ff52f0e0043b-kube-api-access-v5s9x\") pod \"klusterlet-addon-workmgr-69484f69f8-6tzjr\" (UID: \"785a605a-5323-4c5d-934c-ff52f0e0043b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69484f69f8-6tzjr" Sep 10 03:49:41.002593 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:41.002597 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vnh2\" (UniqueName: \"kubernetes.io/projected/005bbc1c-6e3d-4b66-b0db-4af80d7e525a-kube-api-access-5vnh2\") pod \"cluster-proxy-proxy-agent-5d9676fc7b-6zjvl\" (UID: \"005bbc1c-6e3d-4b66-b0db-4af80d7e525a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d9676fc7b-6zjvl" Sep 10 03:49:41.002832 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:41.002647 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/785a605a-5323-4c5d-934c-ff52f0e0043b-tmp\") pod \"klusterlet-addon-workmgr-69484f69f8-6tzjr\" (UID: \"785a605a-5323-4c5d-934c-ff52f0e0043b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69484f69f8-6tzjr" Sep 10 03:49:41.002832 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:41.002674 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/005bbc1c-6e3d-4b66-b0db-4af80d7e525a-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5d9676fc7b-6zjvl\" (UID: \"005bbc1c-6e3d-4b66-b0db-4af80d7e525a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d9676fc7b-6zjvl" Sep 10 03:49:41.002832 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:41.002699 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/005bbc1c-6e3d-4b66-b0db-4af80d7e525a-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5d9676fc7b-6zjvl\" (UID: \"005bbc1c-6e3d-4b66-b0db-4af80d7e525a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d9676fc7b-6zjvl" Sep 10 03:49:41.003008 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:41.002852 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/005bbc1c-6e3d-4b66-b0db-4af80d7e525a-ca\") pod \"cluster-proxy-proxy-agent-5d9676fc7b-6zjvl\" (UID: \"005bbc1c-6e3d-4b66-b0db-4af80d7e525a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d9676fc7b-6zjvl" Sep 10 03:49:41.003008 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:41.002906 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/005bbc1c-6e3d-4b66-b0db-4af80d7e525a-hub\") pod \"cluster-proxy-proxy-agent-5d9676fc7b-6zjvl\" (UID: \"005bbc1c-6e3d-4b66-b0db-4af80d7e525a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d9676fc7b-6zjvl" Sep 10 03:49:41.003008 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:41.002936 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/005bbc1c-6e3d-4b66-b0db-4af80d7e525a-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5d9676fc7b-6zjvl\" (UID: \"005bbc1c-6e3d-4b66-b0db-4af80d7e525a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d9676fc7b-6zjvl" Sep 10 03:49:41.003008 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:41.002997 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/785a605a-5323-4c5d-934c-ff52f0e0043b-klusterlet-config\") pod \"klusterlet-addon-workmgr-69484f69f8-6tzjr\" (UID: \"785a605a-5323-4c5d-934c-ff52f0e0043b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69484f69f8-6tzjr" Sep 10 03:49:41.003891 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:41.003866 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/005bbc1c-6e3d-4b66-b0db-4af80d7e525a-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5d9676fc7b-6zjvl\" (UID: \"005bbc1c-6e3d-4b66-b0db-4af80d7e525a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d9676fc7b-6zjvl" Sep 10 03:49:41.004090 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:41.003869 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/785a605a-5323-4c5d-934c-ff52f0e0043b-tmp\") pod \"klusterlet-addon-workmgr-69484f69f8-6tzjr\" (UID: \"785a605a-5323-4c5d-934c-ff52f0e0043b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69484f69f8-6tzjr" Sep 10 03:49:41.005700 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:41.005674 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/005bbc1c-6e3d-4b66-b0db-4af80d7e525a-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5d9676fc7b-6zjvl\" (UID: \"005bbc1c-6e3d-4b66-b0db-4af80d7e525a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d9676fc7b-6zjvl" Sep 10 03:49:41.005810 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:41.005789 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/005bbc1c-6e3d-4b66-b0db-4af80d7e525a-ca\") pod \"cluster-proxy-proxy-agent-5d9676fc7b-6zjvl\" (UID: \"005bbc1c-6e3d-4b66-b0db-4af80d7e525a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d9676fc7b-6zjvl" Sep 10 03:49:41.005924 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:41.005906 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/785a605a-5323-4c5d-934c-ff52f0e0043b-klusterlet-config\") pod \"klusterlet-addon-workmgr-69484f69f8-6tzjr\" (UID: \"785a605a-5323-4c5d-934c-ff52f0e0043b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69484f69f8-6tzjr" Sep 10 03:49:41.006067 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:41.006050 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/005bbc1c-6e3d-4b66-b0db-4af80d7e525a-hub\") pod \"cluster-proxy-proxy-agent-5d9676fc7b-6zjvl\" (UID: \"005bbc1c-6e3d-4b66-b0db-4af80d7e525a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d9676fc7b-6zjvl" Sep 10 03:49:41.006125 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:41.006075 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/005bbc1c-6e3d-4b66-b0db-4af80d7e525a-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5d9676fc7b-6zjvl\" (UID: \"005bbc1c-6e3d-4b66-b0db-4af80d7e525a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d9676fc7b-6zjvl" Sep 10 03:49:41.010815 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:41.010792 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vnh2\" (UniqueName: \"kubernetes.io/projected/005bbc1c-6e3d-4b66-b0db-4af80d7e525a-kube-api-access-5vnh2\") pod \"cluster-proxy-proxy-agent-5d9676fc7b-6zjvl\" (UID: \"005bbc1c-6e3d-4b66-b0db-4af80d7e525a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d9676fc7b-6zjvl" Sep 10 03:49:41.011077 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:41.011061 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5s9x\" (UniqueName: \"kubernetes.io/projected/785a605a-5323-4c5d-934c-ff52f0e0043b-kube-api-access-v5s9x\") pod \"klusterlet-addon-workmgr-69484f69f8-6tzjr\" (UID: \"785a605a-5323-4c5d-934c-ff52f0e0043b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69484f69f8-6tzjr" Sep 10 03:49:41.155925 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:41.155844 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69484f69f8-6tzjr" Sep 10 03:49:41.199114 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:41.199081 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d9676fc7b-6zjvl" Sep 10 03:49:41.425029 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:41.424934 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-69484f69f8-6tzjr"] Sep 10 03:49:41.427792 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:49:41.427765 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod785a605a_5323_4c5d_934c_ff52f0e0043b.slice/crio-c61e15035afce0d105cccccf475f67d405e502402bc24feda8a6a74fdde838f9 WatchSource:0}: Error finding container c61e15035afce0d105cccccf475f67d405e502402bc24feda8a6a74fdde838f9: Status 404 returned error can't find the container with id c61e15035afce0d105cccccf475f67d405e502402bc24feda8a6a74fdde838f9 Sep 10 03:49:41.470440 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:41.470410 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d9676fc7b-6zjvl"] Sep 10 03:49:41.473558 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:49:41.473534 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod005bbc1c_6e3d_4b66_b0db_4af80d7e525a.slice/crio-9b8cd750a36a5d664e6f7d6e5702ffdb44a248ed0d4d8be9ee39bab0b60bb9e8 WatchSource:0}: Error finding container 9b8cd750a36a5d664e6f7d6e5702ffdb44a248ed0d4d8be9ee39bab0b60bb9e8: Status 404 returned error can't find the container with id 9b8cd750a36a5d664e6f7d6e5702ffdb44a248ed0d4d8be9ee39bab0b60bb9e8 Sep 10 03:49:42.265926 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:42.265889 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d9676fc7b-6zjvl" event={"ID":"005bbc1c-6e3d-4b66-b0db-4af80d7e525a","Type":"ContainerStarted","Data":"9b8cd750a36a5d664e6f7d6e5702ffdb44a248ed0d4d8be9ee39bab0b60bb9e8"} Sep 10 03:49:42.267903 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:42.267873 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69484f69f8-6tzjr" event={"ID":"785a605a-5323-4c5d-934c-ff52f0e0043b","Type":"ContainerStarted","Data":"c61e15035afce0d105cccccf475f67d405e502402bc24feda8a6a74fdde838f9"} Sep 10 03:49:44.731319 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:44.731274 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/184e14a2-418b-41da-8635-23ea4c965dbc-metrics-tls\") pod \"dns-default-kzds7\" (UID: \"184e14a2-418b-41da-8635-23ea4c965dbc\") " pod="openshift-dns/dns-default-kzds7" Sep 10 03:49:44.731797 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:49:44.731412 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 03:49:44.731797 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:49:44.731485 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/184e14a2-418b-41da-8635-23ea4c965dbc-metrics-tls podName:184e14a2-418b-41da-8635-23ea4c965dbc nodeName:}" failed. No retries permitted until 2026-09-10 03:50:48.731465541 +0000 UTC m=+162.256844627 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/184e14a2-418b-41da-8635-23ea4c965dbc-metrics-tls") pod "dns-default-kzds7" (UID: "184e14a2-418b-41da-8635-23ea4c965dbc") : secret "dns-default-metrics-tls" not found Sep 10 03:49:45.277463 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:45.277375 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69484f69f8-6tzjr" event={"ID":"785a605a-5323-4c5d-934c-ff52f0e0043b","Type":"ContainerStarted","Data":"65af673b517dc4488f9f874f934a0a6935b11af1f8f178fac85d803fdf4b9b06"} Sep 10 03:49:45.277610 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:45.277588 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69484f69f8-6tzjr" Sep 10 03:49:45.279060 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:45.279038 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69484f69f8-6tzjr" Sep 10 03:49:45.292264 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:45.292226 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69484f69f8-6tzjr" podStartSLOduration=1.7020985419999999 podStartE2EDuration="5.292213948s" totalImagesPullingTime="3.590115406s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:49:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:49:45.291122547 +0000 UTC m=+98.816501651" watchObservedRunningTime="2026-09-10 03:49:45.292213948 +0000 UTC m=+98.817593053" Sep 10 03:49:49.209903 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:49.209319 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-7r9qf" Sep 10 03:49:52.300729 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:52.300692 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d9676fc7b-6zjvl" event={"ID":"005bbc1c-6e3d-4b66-b0db-4af80d7e525a","Type":"ContainerStarted","Data":"9e517dd9af5e7e4d21cf27224bf00e45ec196163e0b8cd6ef5cabb295b892683"} Sep 10 03:49:52.300729 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:52.300735 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d9676fc7b-6zjvl" event={"ID":"005bbc1c-6e3d-4b66-b0db-4af80d7e525a","Type":"ContainerStarted","Data":"7e0d0b4a7b9abc529a7b25a28a2cc2d0c30d9d0bbb41a52eafd7eba3868ff281"} Sep 10 03:49:52.301284 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:52.300749 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d9676fc7b-6zjvl" event={"ID":"005bbc1c-6e3d-4b66-b0db-4af80d7e525a","Type":"ContainerStarted","Data":"7e9c841f42efdcd668af05a436fa162491fd46afa7777e1e42f5c1b4aa1310cf"} Sep 10 03:49:52.318513 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:49:52.318461 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d9676fc7b-6zjvl" podStartSLOduration=2.588509037 podStartE2EDuration="12.318447082s" totalImagesPullingTime="9.729938045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:49:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:49:52.316608226 +0000 UTC m=+105.841987331" watchObservedRunningTime="2026-09-10 03:49:52.318447082 +0000 UTC m=+105.843826202" Sep 10 03:50:16.740915 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:50:16.740873 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fca9ba6d-4934-47e4-8be5-4296c7a6c530-metrics-certs\") pod \"network-metrics-daemon-htpt9\" (UID: \"fca9ba6d-4934-47e4-8be5-4296c7a6c530\") " pod="openshift-multus/network-metrics-daemon-htpt9" Sep 10 03:50:16.741411 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:50:16.741004 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 10 03:50:16.741411 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:50:16.741070 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fca9ba6d-4934-47e4-8be5-4296c7a6c530-metrics-certs podName:fca9ba6d-4934-47e4-8be5-4296c7a6c530 nodeName:}" failed. No retries permitted until 2026-09-10 03:52:18.741056184 +0000 UTC m=+252.266435267 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fca9ba6d-4934-47e4-8be5-4296c7a6c530-metrics-certs") pod "network-metrics-daemon-htpt9" (UID: "fca9ba6d-4934-47e4-8be5-4296c7a6c530") : secret "metrics-daemon-secret" not found Sep 10 03:50:43.879824 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:50:43.879781 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-kzds7" podUID="184e14a2-418b-41da-8635-23ea4c965dbc" Sep 10 03:50:43.956058 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:50:43.956025 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-htpt9" podUID="fca9ba6d-4934-47e4-8be5-4296c7a6c530" Sep 10 03:50:44.447992 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:50:44.447943 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kzds7" Sep 10 03:50:48.765604 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:50:48.765561 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/184e14a2-418b-41da-8635-23ea4c965dbc-metrics-tls\") pod \"dns-default-kzds7\" (UID: \"184e14a2-418b-41da-8635-23ea4c965dbc\") " pod="openshift-dns/dns-default-kzds7" Sep 10 03:50:48.766017 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:50:48.765676 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 03:50:48.766017 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:50:48.765728 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/184e14a2-418b-41da-8635-23ea4c965dbc-metrics-tls podName:184e14a2-418b-41da-8635-23ea4c965dbc nodeName:}" failed. No retries permitted until 2026-09-10 03:52:50.765715569 +0000 UTC m=+284.291094652 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/184e14a2-418b-41da-8635-23ea4c965dbc-metrics-tls") pod "dns-default-kzds7" (UID: "184e14a2-418b-41da-8635-23ea4c965dbc") : secret "dns-default-metrics-tls" not found Sep 10 03:50:58.937639 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:50:58.937545 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htpt9" Sep 10 03:51:45.278784 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:51:45.278731 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69484f69f8-6tzjr" podUID="785a605a-5323-4c5d-934c-ff52f0e0043b" containerName="acm-agent" probeResult="failure" output="Get \"http://10.134.0.8:8000/readyz\": dial tcp 10.134.0.8:8000: connect: connection refused" Sep 10 03:51:45.624737 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:51:45.624706 2581 generic.go:356] "Generic (PLEG): container finished" podID="785a605a-5323-4c5d-934c-ff52f0e0043b" containerID="65af673b517dc4488f9f874f934a0a6935b11af1f8f178fac85d803fdf4b9b06" exitCode=1 Sep 10 03:51:45.624891 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:51:45.624768 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69484f69f8-6tzjr" event={"ID":"785a605a-5323-4c5d-934c-ff52f0e0043b","Type":"ContainerDied","Data":"65af673b517dc4488f9f874f934a0a6935b11af1f8f178fac85d803fdf4b9b06"} Sep 10 03:51:45.625089 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:51:45.625074 2581 scope.go:117] "RemoveContainer" containerID="65af673b517dc4488f9f874f934a0a6935b11af1f8f178fac85d803fdf4b9b06" Sep 10 03:51:46.629789 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:51:46.629758 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69484f69f8-6tzjr" event={"ID":"785a605a-5323-4c5d-934c-ff52f0e0043b","Type":"ContainerStarted","Data":"57affc4c7ecdedbd1c05c8a5e5f8f9139e0743b30665bd0ab29faa4178ca1896"} Sep 10 03:51:46.630187 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:51:46.630038 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69484f69f8-6tzjr" Sep 10 03:51:46.630608 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:51:46.630593 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69484f69f8-6tzjr" Sep 10 03:52:18.792333 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:18.792287 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fca9ba6d-4934-47e4-8be5-4296c7a6c530-metrics-certs\") pod \"network-metrics-daemon-htpt9\" (UID: \"fca9ba6d-4934-47e4-8be5-4296c7a6c530\") " pod="openshift-multus/network-metrics-daemon-htpt9" Sep 10 03:52:18.792840 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:52:18.792406 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 10 03:52:18.792840 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:52:18.792454 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fca9ba6d-4934-47e4-8be5-4296c7a6c530-metrics-certs podName:fca9ba6d-4934-47e4-8be5-4296c7a6c530 nodeName:}" failed. No retries permitted until 2026-09-10 03:54:20.792441833 +0000 UTC m=+374.317820916 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fca9ba6d-4934-47e4-8be5-4296c7a6c530-metrics-certs") pod "network-metrics-daemon-htpt9" (UID: "fca9ba6d-4934-47e4-8be5-4296c7a6c530") : secret "metrics-daemon-secret" not found Sep 10 03:52:20.247266 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:20.247233 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-hj47d"] Sep 10 03:52:20.249678 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:20.249660 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hj47d" Sep 10 03:52:20.252498 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:20.252478 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 10 03:52:20.252776 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:20.252756 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 10 03:52:20.252841 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:20.252766 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nhqj9\"" Sep 10 03:52:20.253536 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:20.253521 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 10 03:52:20.253585 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:20.253551 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 10 03:52:20.262725 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:20.262699 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hj47d"] Sep 10 03:52:20.402829 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:20.402793 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4tnk\" (UniqueName: \"kubernetes.io/projected/53b35768-5fa3-41d1-87f2-4fb4f7adc140-kube-api-access-q4tnk\") pod \"insights-runtime-extractor-hj47d\" (UID: \"53b35768-5fa3-41d1-87f2-4fb4f7adc140\") " pod="openshift-insights/insights-runtime-extractor-hj47d" Sep 10 03:52:20.402829 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:20.402833 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/53b35768-5fa3-41d1-87f2-4fb4f7adc140-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hj47d\" (UID: \"53b35768-5fa3-41d1-87f2-4fb4f7adc140\") " pod="openshift-insights/insights-runtime-extractor-hj47d" Sep 10 03:52:20.403050 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:20.402944 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/53b35768-5fa3-41d1-87f2-4fb4f7adc140-data-volume\") pod \"insights-runtime-extractor-hj47d\" (UID: \"53b35768-5fa3-41d1-87f2-4fb4f7adc140\") " pod="openshift-insights/insights-runtime-extractor-hj47d" Sep 10 03:52:20.403050 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:20.403026 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/53b35768-5fa3-41d1-87f2-4fb4f7adc140-crio-socket\") pod \"insights-runtime-extractor-hj47d\" (UID: \"53b35768-5fa3-41d1-87f2-4fb4f7adc140\") " pod="openshift-insights/insights-runtime-extractor-hj47d" Sep 10 03:52:20.403050 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:20.403043 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53b35768-5fa3-41d1-87f2-4fb4f7adc140-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hj47d\" (UID: \"53b35768-5fa3-41d1-87f2-4fb4f7adc140\") " pod="openshift-insights/insights-runtime-extractor-hj47d" Sep 10 03:52:20.504192 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:20.504107 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/53b35768-5fa3-41d1-87f2-4fb4f7adc140-crio-socket\") pod \"insights-runtime-extractor-hj47d\" (UID: \"53b35768-5fa3-41d1-87f2-4fb4f7adc140\") " pod="openshift-insights/insights-runtime-extractor-hj47d" Sep 10 03:52:20.504192 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:20.504156 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53b35768-5fa3-41d1-87f2-4fb4f7adc140-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hj47d\" (UID: \"53b35768-5fa3-41d1-87f2-4fb4f7adc140\") " pod="openshift-insights/insights-runtime-extractor-hj47d" Sep 10 03:52:20.504192 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:20.504188 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4tnk\" (UniqueName: \"kubernetes.io/projected/53b35768-5fa3-41d1-87f2-4fb4f7adc140-kube-api-access-q4tnk\") pod \"insights-runtime-extractor-hj47d\" (UID: \"53b35768-5fa3-41d1-87f2-4fb4f7adc140\") " pod="openshift-insights/insights-runtime-extractor-hj47d" Sep 10 03:52:20.504357 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:20.504207 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/53b35768-5fa3-41d1-87f2-4fb4f7adc140-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hj47d\" (UID: \"53b35768-5fa3-41d1-87f2-4fb4f7adc140\") " pod="openshift-insights/insights-runtime-extractor-hj47d" Sep 10 03:52:20.504357 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:20.504241 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/53b35768-5fa3-41d1-87f2-4fb4f7adc140-data-volume\") pod \"insights-runtime-extractor-hj47d\" (UID: \"53b35768-5fa3-41d1-87f2-4fb4f7adc140\") " pod="openshift-insights/insights-runtime-extractor-hj47d" Sep 10 03:52:20.504357 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:20.504244 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/53b35768-5fa3-41d1-87f2-4fb4f7adc140-crio-socket\") pod \"insights-runtime-extractor-hj47d\" (UID: \"53b35768-5fa3-41d1-87f2-4fb4f7adc140\") " pod="openshift-insights/insights-runtime-extractor-hj47d" Sep 10 03:52:20.504357 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:52:20.504316 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 03:52:20.504357 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:52:20.504347 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 03:52:20.504542 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:52:20.504382 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/53b35768-5fa3-41d1-87f2-4fb4f7adc140-kube-rbac-proxy-cm podName:53b35768-5fa3-41d1-87f2-4fb4f7adc140 nodeName:}" failed. No retries permitted until 2026-09-10 03:52:21.004368615 +0000 UTC m=+254.529747697 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/53b35768-5fa3-41d1-87f2-4fb4f7adc140-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hj47d" (UID: "53b35768-5fa3-41d1-87f2-4fb4f7adc140") : configmap "kube-rbac-proxy" not found Sep 10 03:52:20.504542 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:52:20.504417 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53b35768-5fa3-41d1-87f2-4fb4f7adc140-insights-runtime-extractor-tls podName:53b35768-5fa3-41d1-87f2-4fb4f7adc140 nodeName:}" failed. No retries permitted until 2026-09-10 03:52:21.004399688 +0000 UTC m=+254.529778772 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/53b35768-5fa3-41d1-87f2-4fb4f7adc140-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hj47d" (UID: "53b35768-5fa3-41d1-87f2-4fb4f7adc140") : secret "insights-runtime-extractor-tls" not found Sep 10 03:52:20.504662 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:20.504644 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/53b35768-5fa3-41d1-87f2-4fb4f7adc140-data-volume\") pod \"insights-runtime-extractor-hj47d\" (UID: \"53b35768-5fa3-41d1-87f2-4fb4f7adc140\") " pod="openshift-insights/insights-runtime-extractor-hj47d" Sep 10 03:52:20.515201 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:20.515172 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4tnk\" (UniqueName: \"kubernetes.io/projected/53b35768-5fa3-41d1-87f2-4fb4f7adc140-kube-api-access-q4tnk\") pod \"insights-runtime-extractor-hj47d\" (UID: \"53b35768-5fa3-41d1-87f2-4fb4f7adc140\") " pod="openshift-insights/insights-runtime-extractor-hj47d" Sep 10 03:52:21.007511 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:21.007477 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53b35768-5fa3-41d1-87f2-4fb4f7adc140-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hj47d\" (UID: \"53b35768-5fa3-41d1-87f2-4fb4f7adc140\") " pod="openshift-insights/insights-runtime-extractor-hj47d" Sep 10 03:52:21.007686 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:21.007526 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/53b35768-5fa3-41d1-87f2-4fb4f7adc140-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hj47d\" (UID: \"53b35768-5fa3-41d1-87f2-4fb4f7adc140\") " pod="openshift-insights/insights-runtime-extractor-hj47d" Sep 10 03:52:21.007686 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:52:21.007596 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 03:52:21.007686 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:52:21.007639 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/53b35768-5fa3-41d1-87f2-4fb4f7adc140-kube-rbac-proxy-cm podName:53b35768-5fa3-41d1-87f2-4fb4f7adc140 nodeName:}" failed. No retries permitted until 2026-09-10 03:52:22.007626933 +0000 UTC m=+255.533006016 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/53b35768-5fa3-41d1-87f2-4fb4f7adc140-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hj47d" (UID: "53b35768-5fa3-41d1-87f2-4fb4f7adc140") : configmap "kube-rbac-proxy" not found Sep 10 03:52:21.007686 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:52:21.007646 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 03:52:21.007865 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:52:21.007720 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53b35768-5fa3-41d1-87f2-4fb4f7adc140-insights-runtime-extractor-tls podName:53b35768-5fa3-41d1-87f2-4fb4f7adc140 nodeName:}" failed. No retries permitted until 2026-09-10 03:52:22.007699504 +0000 UTC m=+255.533078587 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/53b35768-5fa3-41d1-87f2-4fb4f7adc140-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hj47d" (UID: "53b35768-5fa3-41d1-87f2-4fb4f7adc140") : secret "insights-runtime-extractor-tls" not found Sep 10 03:52:22.012267 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:22.012221 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/53b35768-5fa3-41d1-87f2-4fb4f7adc140-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hj47d\" (UID: \"53b35768-5fa3-41d1-87f2-4fb4f7adc140\") " pod="openshift-insights/insights-runtime-extractor-hj47d" Sep 10 03:52:22.012764 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:22.012295 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53b35768-5fa3-41d1-87f2-4fb4f7adc140-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hj47d\" (UID: \"53b35768-5fa3-41d1-87f2-4fb4f7adc140\") " pod="openshift-insights/insights-runtime-extractor-hj47d" Sep 10 03:52:22.012764 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:52:22.012351 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 03:52:22.012764 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:52:22.012400 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 03:52:22.012764 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:52:22.012428 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/53b35768-5fa3-41d1-87f2-4fb4f7adc140-kube-rbac-proxy-cm podName:53b35768-5fa3-41d1-87f2-4fb4f7adc140 nodeName:}" failed. No retries permitted until 2026-09-10 03:52:24.012407306 +0000 UTC m=+257.537786391 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/53b35768-5fa3-41d1-87f2-4fb4f7adc140-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hj47d" (UID: "53b35768-5fa3-41d1-87f2-4fb4f7adc140") : configmap "kube-rbac-proxy" not found Sep 10 03:52:22.012764 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:52:22.012447 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53b35768-5fa3-41d1-87f2-4fb4f7adc140-insights-runtime-extractor-tls podName:53b35768-5fa3-41d1-87f2-4fb4f7adc140 nodeName:}" failed. No retries permitted until 2026-09-10 03:52:24.012436549 +0000 UTC m=+257.537815632 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/53b35768-5fa3-41d1-87f2-4fb4f7adc140-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hj47d" (UID: "53b35768-5fa3-41d1-87f2-4fb4f7adc140") : secret "insights-runtime-extractor-tls" not found Sep 10 03:52:22.202577 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:22.202552 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7lpdd_29111c2f-bef9-4076-9014-7f041915d6df/dns-node-resolver/0.log" Sep 10 03:52:23.404038 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:23.404010 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-r58nd_b4c0f4fb-e2d6-4ed5-969e-ec1ff4aaff92/node-ca/0.log" Sep 10 03:52:23.802509 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:23.802485 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-cmdjp_59ba01f7-c3bf-4ccb-8a19-f69bbda08d5f/serve-healthcheck-canary/0.log" Sep 10 03:52:24.027037 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:24.027003 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53b35768-5fa3-41d1-87f2-4fb4f7adc140-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hj47d\" (UID: \"53b35768-5fa3-41d1-87f2-4fb4f7adc140\") " pod="openshift-insights/insights-runtime-extractor-hj47d" Sep 10 03:52:24.027272 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:24.027054 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/53b35768-5fa3-41d1-87f2-4fb4f7adc140-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hj47d\" (UID: \"53b35768-5fa3-41d1-87f2-4fb4f7adc140\") " pod="openshift-insights/insights-runtime-extractor-hj47d" Sep 10 03:52:24.027272 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:52:24.027122 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 03:52:24.027272 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:52:24.027160 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 03:52:24.027272 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:52:24.027175 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/53b35768-5fa3-41d1-87f2-4fb4f7adc140-kube-rbac-proxy-cm podName:53b35768-5fa3-41d1-87f2-4fb4f7adc140 nodeName:}" failed. No retries permitted until 2026-09-10 03:52:28.0271623 +0000 UTC m=+261.552541384 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/53b35768-5fa3-41d1-87f2-4fb4f7adc140-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hj47d" (UID: "53b35768-5fa3-41d1-87f2-4fb4f7adc140") : configmap "kube-rbac-proxy" not found Sep 10 03:52:24.027272 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:52:24.027210 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53b35768-5fa3-41d1-87f2-4fb4f7adc140-insights-runtime-extractor-tls podName:53b35768-5fa3-41d1-87f2-4fb4f7adc140 nodeName:}" failed. No retries permitted until 2026-09-10 03:52:28.027197623 +0000 UTC m=+261.552576707 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/53b35768-5fa3-41d1-87f2-4fb4f7adc140-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hj47d" (UID: "53b35768-5fa3-41d1-87f2-4fb4f7adc140") : secret "insights-runtime-extractor-tls" not found Sep 10 03:52:28.056377 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:28.056341 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53b35768-5fa3-41d1-87f2-4fb4f7adc140-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hj47d\" (UID: \"53b35768-5fa3-41d1-87f2-4fb4f7adc140\") " pod="openshift-insights/insights-runtime-extractor-hj47d" Sep 10 03:52:28.056852 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:28.056400 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/53b35768-5fa3-41d1-87f2-4fb4f7adc140-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hj47d\" (UID: \"53b35768-5fa3-41d1-87f2-4fb4f7adc140\") " pod="openshift-insights/insights-runtime-extractor-hj47d" Sep 10 03:52:28.056852 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:52:28.056478 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 03:52:28.056852 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:52:28.056515 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 03:52:28.056852 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:52:28.056556 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/53b35768-5fa3-41d1-87f2-4fb4f7adc140-kube-rbac-proxy-cm podName:53b35768-5fa3-41d1-87f2-4fb4f7adc140 nodeName:}" failed. No retries permitted until 2026-09-10 03:52:36.056529212 +0000 UTC m=+269.581908298 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/53b35768-5fa3-41d1-87f2-4fb4f7adc140-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hj47d" (UID: "53b35768-5fa3-41d1-87f2-4fb4f7adc140") : configmap "kube-rbac-proxy" not found Sep 10 03:52:28.056852 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:52:28.056576 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53b35768-5fa3-41d1-87f2-4fb4f7adc140-insights-runtime-extractor-tls podName:53b35768-5fa3-41d1-87f2-4fb4f7adc140 nodeName:}" failed. No retries permitted until 2026-09-10 03:52:36.05656325 +0000 UTC m=+269.581942352 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/53b35768-5fa3-41d1-87f2-4fb4f7adc140-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hj47d" (UID: "53b35768-5fa3-41d1-87f2-4fb4f7adc140") : secret "insights-runtime-extractor-tls" not found Sep 10 03:52:36.114867 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:36.114822 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53b35768-5fa3-41d1-87f2-4fb4f7adc140-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hj47d\" (UID: \"53b35768-5fa3-41d1-87f2-4fb4f7adc140\") " pod="openshift-insights/insights-runtime-extractor-hj47d" Sep 10 03:52:36.115353 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:36.114884 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/53b35768-5fa3-41d1-87f2-4fb4f7adc140-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hj47d\" (UID: \"53b35768-5fa3-41d1-87f2-4fb4f7adc140\") " pod="openshift-insights/insights-runtime-extractor-hj47d" Sep 10 03:52:36.115353 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:52:36.114990 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 03:52:36.115353 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:52:36.115067 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53b35768-5fa3-41d1-87f2-4fb4f7adc140-insights-runtime-extractor-tls podName:53b35768-5fa3-41d1-87f2-4fb4f7adc140 nodeName:}" failed. No retries permitted until 2026-09-10 03:52:52.115052287 +0000 UTC m=+285.640431374 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/53b35768-5fa3-41d1-87f2-4fb4f7adc140-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hj47d" (UID: "53b35768-5fa3-41d1-87f2-4fb4f7adc140") : secret "insights-runtime-extractor-tls" not found Sep 10 03:52:36.115461 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:36.115374 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/53b35768-5fa3-41d1-87f2-4fb4f7adc140-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hj47d\" (UID: \"53b35768-5fa3-41d1-87f2-4fb4f7adc140\") " pod="openshift-insights/insights-runtime-extractor-hj47d" Sep 10 03:52:41.200359 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:41.200322 2581 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d9676fc7b-6zjvl" podUID="005bbc1c-6e3d-4b66-b0db-4af80d7e525a" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Sep 10 03:52:47.448483 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:52:47.448425 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-kzds7" podUID="184e14a2-418b-41da-8635-23ea4c965dbc" Sep 10 03:52:47.806718 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:47.806684 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kzds7" Sep 10 03:52:49.919376 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:49.919345 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-rcplh"] Sep 10 03:52:49.922260 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:49.922244 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-rcplh" Sep 10 03:52:49.924401 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:49.924385 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-bb76w\"" Sep 10 03:52:49.924482 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:49.924447 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Sep 10 03:52:49.933123 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:49.931861 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-rcplh"] Sep 10 03:52:50.005041 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:50.004965 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/853ceb7f-4c96-4f16-af84-d6f89525afcb-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-rcplh\" (UID: \"853ceb7f-4c96-4f16-af84-d6f89525afcb\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-rcplh" Sep 10 03:52:50.106396 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:50.106355 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/853ceb7f-4c96-4f16-af84-d6f89525afcb-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-rcplh\" (UID: \"853ceb7f-4c96-4f16-af84-d6f89525afcb\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-rcplh" Sep 10 03:52:50.106552 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:52:50.106503 2581 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Sep 10 03:52:50.106597 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:52:50.106570 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/853ceb7f-4c96-4f16-af84-d6f89525afcb-tls-certificates podName:853ceb7f-4c96-4f16-af84-d6f89525afcb nodeName:}" failed. No retries permitted until 2026-09-10 03:52:50.606553651 +0000 UTC m=+284.131932734 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/853ceb7f-4c96-4f16-af84-d6f89525afcb-tls-certificates") pod "prometheus-operator-admission-webhook-78f599db79-rcplh" (UID: "853ceb7f-4c96-4f16-af84-d6f89525afcb") : secret "prometheus-operator-admission-webhook-tls" not found Sep 10 03:52:50.610076 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:50.610036 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/853ceb7f-4c96-4f16-af84-d6f89525afcb-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-rcplh\" (UID: \"853ceb7f-4c96-4f16-af84-d6f89525afcb\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-rcplh" Sep 10 03:52:50.612488 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:50.612462 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/853ceb7f-4c96-4f16-af84-d6f89525afcb-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-rcplh\" (UID: \"853ceb7f-4c96-4f16-af84-d6f89525afcb\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-rcplh" Sep 10 03:52:50.812178 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:50.812129 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/184e14a2-418b-41da-8635-23ea4c965dbc-metrics-tls\") pod \"dns-default-kzds7\" (UID: \"184e14a2-418b-41da-8635-23ea4c965dbc\") " pod="openshift-dns/dns-default-kzds7" Sep 10 03:52:50.814883 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:50.814859 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/184e14a2-418b-41da-8635-23ea4c965dbc-metrics-tls\") pod \"dns-default-kzds7\" (UID: \"184e14a2-418b-41da-8635-23ea4c965dbc\") " pod="openshift-dns/dns-default-kzds7" Sep 10 03:52:50.831813 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:50.831790 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-rcplh" Sep 10 03:52:51.096870 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:51.096839 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-rcplh"] Sep 10 03:52:51.099845 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:52:51.099819 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod853ceb7f_4c96_4f16_af84_d6f89525afcb.slice/crio-4cb30e6c1895b1ca69ac26874965ca03075e6b92b99b80c13af4df9cb0e66b67 WatchSource:0}: Error finding container 4cb30e6c1895b1ca69ac26874965ca03075e6b92b99b80c13af4df9cb0e66b67: Status 404 returned error can't find the container with id 4cb30e6c1895b1ca69ac26874965ca03075e6b92b99b80c13af4df9cb0e66b67 Sep 10 03:52:51.109342 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:51.109319 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bt2fn\"" Sep 10 03:52:51.117751 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:51.117732 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kzds7" Sep 10 03:52:51.200601 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:51.200568 2581 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d9676fc7b-6zjvl" podUID="005bbc1c-6e3d-4b66-b0db-4af80d7e525a" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Sep 10 03:52:51.391897 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:51.391811 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-kzds7"] Sep 10 03:52:51.395305 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:52:51.395277 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod184e14a2_418b_41da_8635_23ea4c965dbc.slice/crio-41aa62941068ca5a118532521b94761a5151f12bf175736e1c1a1320363a96db WatchSource:0}: Error finding container 41aa62941068ca5a118532521b94761a5151f12bf175736e1c1a1320363a96db: Status 404 returned error can't find the container with id 41aa62941068ca5a118532521b94761a5151f12bf175736e1c1a1320363a96db Sep 10 03:52:51.820046 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:51.820008 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kzds7" event={"ID":"184e14a2-418b-41da-8635-23ea4c965dbc","Type":"ContainerStarted","Data":"41aa62941068ca5a118532521b94761a5151f12bf175736e1c1a1320363a96db"} Sep 10 03:52:51.821597 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:51.821556 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-rcplh" event={"ID":"853ceb7f-4c96-4f16-af84-d6f89525afcb","Type":"ContainerStarted","Data":"4cb30e6c1895b1ca69ac26874965ca03075e6b92b99b80c13af4df9cb0e66b67"} Sep 10 03:52:52.121839 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:52.121738 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53b35768-5fa3-41d1-87f2-4fb4f7adc140-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hj47d\" (UID: \"53b35768-5fa3-41d1-87f2-4fb4f7adc140\") " pod="openshift-insights/insights-runtime-extractor-hj47d" Sep 10 03:52:52.124538 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:52.124515 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53b35768-5fa3-41d1-87f2-4fb4f7adc140-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hj47d\" (UID: \"53b35768-5fa3-41d1-87f2-4fb4f7adc140\") " pod="openshift-insights/insights-runtime-extractor-hj47d" Sep 10 03:52:52.360831 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:52.360813 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hj47d" Sep 10 03:52:52.644557 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:52.644469 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hj47d"] Sep 10 03:52:52.827282 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:52.827246 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-rcplh" event={"ID":"853ceb7f-4c96-4f16-af84-d6f89525afcb","Type":"ContainerStarted","Data":"36b3574f7cfac4d65886638235bba298a2755cc782754ccc66f5539fe5d6f753"} Sep 10 03:52:52.827604 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:52.827581 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-rcplh" Sep 10 03:52:52.833343 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:52.833316 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-rcplh" Sep 10 03:52:52.841962 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:52.841795 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-rcplh" podStartSLOduration=2.589329288 podStartE2EDuration="3.84175405s" totalImagesPullingTime="1.252424762s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:52:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:52:52.841717539 +0000 UTC m=+286.367096645" watchObservedRunningTime="2026-09-10 03:52:52.84175405 +0000 UTC m=+286.367133136" Sep 10 03:52:52.923358 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:52:52.923277 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53b35768_5fa3_41d1_87f2_4fb4f7adc140.slice/crio-90458ae9dfe882d6cfadae2a5e0da33c16224d71f31e8c4323da53536b14f241 WatchSource:0}: Error finding container 90458ae9dfe882d6cfadae2a5e0da33c16224d71f31e8c4323da53536b14f241: Status 404 returned error can't find the container with id 90458ae9dfe882d6cfadae2a5e0da33c16224d71f31e8c4323da53536b14f241 Sep 10 03:52:53.833031 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:53.832990 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kzds7" event={"ID":"184e14a2-418b-41da-8635-23ea4c965dbc","Type":"ContainerStarted","Data":"d04d319795394ce39ee81378275f89b37e018eaaa3ee5086b7cbaeb573387ce0"} Sep 10 03:52:53.833507 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:53.833046 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kzds7" event={"ID":"184e14a2-418b-41da-8635-23ea4c965dbc","Type":"ContainerStarted","Data":"7f2bd614bc538531c88985200c6c7db546fcb0fc540745e440acd52e5bf8bec4"} Sep 10 03:52:53.833507 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:53.833104 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-kzds7" Sep 10 03:52:53.834771 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:53.834751 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hj47d" event={"ID":"53b35768-5fa3-41d1-87f2-4fb4f7adc140","Type":"ContainerStarted","Data":"b0bd8f45baaa61a3887c0b75fe4ce47d45aaa10b9b77ed08f6989ef0814b9801"} Sep 10 03:52:53.834869 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:53.834792 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hj47d" event={"ID":"53b35768-5fa3-41d1-87f2-4fb4f7adc140","Type":"ContainerStarted","Data":"0f8a1d1eb5b63b68190a38cd6f9dbb334a5fcaad61cf6a01328a37b0712ae2f9"} Sep 10 03:52:53.834869 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:53.834815 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hj47d" event={"ID":"53b35768-5fa3-41d1-87f2-4fb4f7adc140","Type":"ContainerStarted","Data":"90458ae9dfe882d6cfadae2a5e0da33c16224d71f31e8c4323da53536b14f241"} Sep 10 03:52:53.849526 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:53.849478 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-kzds7" podStartSLOduration=252.275244063 podStartE2EDuration="4m13.849462862s" totalImagesPullingTime="1.574218799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:48:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:52:53.849145246 +0000 UTC m=+287.374524346" watchObservedRunningTime="2026-09-10 03:52:53.849462862 +0000 UTC m=+287.374841967" Sep 10 03:52:55.843888 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:55.843852 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hj47d" event={"ID":"53b35768-5fa3-41d1-87f2-4fb4f7adc140","Type":"ContainerStarted","Data":"fa31b2c7d23083c75486ab5d7db2c3ea9e9db31e83aed0e7c35cd08290c95f69"} Sep 10 03:52:55.863339 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:55.862450 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-hj47d" podStartSLOduration=33.928648259 podStartE2EDuration="35.862431043s" totalImagesPullingTime="1.933782784s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 03:52:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:52:55.861530512 +0000 UTC m=+289.386909618" watchObservedRunningTime="2026-09-10 03:52:55.862431043 +0000 UTC m=+289.387810149" Sep 10 03:52:58.350612 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:58.350580 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-dsxrz"] Sep 10 03:52:58.353743 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:58.353723 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dsxrz" Sep 10 03:52:58.356155 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:58.356134 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 10 03:52:58.356746 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:58.356730 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 10 03:52:58.357227 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:58.357208 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 10 03:52:58.357379 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:58.357354 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 10 03:52:58.357581 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:58.357562 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 10 03:52:58.357646 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:58.357571 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 10 03:52:58.357646 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:58.357607 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-s6wbd\"" Sep 10 03:52:58.464046 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:58.464015 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/df73b602-cb9a-4072-bd26-88bef578eef5-node-exporter-textfile\") pod \"node-exporter-dsxrz\" (UID: \"df73b602-cb9a-4072-bd26-88bef578eef5\") " pod="openshift-monitoring/node-exporter-dsxrz" Sep 10 03:52:58.464046 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:58.464048 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zh4th\" (UniqueName: \"kubernetes.io/projected/df73b602-cb9a-4072-bd26-88bef578eef5-kube-api-access-zh4th\") pod \"node-exporter-dsxrz\" (UID: \"df73b602-cb9a-4072-bd26-88bef578eef5\") " pod="openshift-monitoring/node-exporter-dsxrz" Sep 10 03:52:58.464291 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:58.464082 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/df73b602-cb9a-4072-bd26-88bef578eef5-node-exporter-accelerators-collector-config\") pod \"node-exporter-dsxrz\" (UID: \"df73b602-cb9a-4072-bd26-88bef578eef5\") " pod="openshift-monitoring/node-exporter-dsxrz" Sep 10 03:52:58.464291 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:58.464101 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/df73b602-cb9a-4072-bd26-88bef578eef5-metrics-client-ca\") pod \"node-exporter-dsxrz\" (UID: \"df73b602-cb9a-4072-bd26-88bef578eef5\") " pod="openshift-monitoring/node-exporter-dsxrz" Sep 10 03:52:58.464291 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:58.464127 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/df73b602-cb9a-4072-bd26-88bef578eef5-root\") pod \"node-exporter-dsxrz\" (UID: \"df73b602-cb9a-4072-bd26-88bef578eef5\") " pod="openshift-monitoring/node-exporter-dsxrz" Sep 10 03:52:58.464291 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:58.464193 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/df73b602-cb9a-4072-bd26-88bef578eef5-sys\") pod \"node-exporter-dsxrz\" (UID: \"df73b602-cb9a-4072-bd26-88bef578eef5\") " pod="openshift-monitoring/node-exporter-dsxrz" Sep 10 03:52:58.464291 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:58.464231 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/df73b602-cb9a-4072-bd26-88bef578eef5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dsxrz\" (UID: \"df73b602-cb9a-4072-bd26-88bef578eef5\") " pod="openshift-monitoring/node-exporter-dsxrz" Sep 10 03:52:58.464291 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:58.464248 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/df73b602-cb9a-4072-bd26-88bef578eef5-node-exporter-tls\") pod \"node-exporter-dsxrz\" (UID: \"df73b602-cb9a-4072-bd26-88bef578eef5\") " pod="openshift-monitoring/node-exporter-dsxrz" Sep 10 03:52:58.464291 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:58.464270 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/df73b602-cb9a-4072-bd26-88bef578eef5-node-exporter-wtmp\") pod \"node-exporter-dsxrz\" (UID: \"df73b602-cb9a-4072-bd26-88bef578eef5\") " pod="openshift-monitoring/node-exporter-dsxrz" Sep 10 03:52:58.565389 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:58.565355 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh4th\" (UniqueName: \"kubernetes.io/projected/df73b602-cb9a-4072-bd26-88bef578eef5-kube-api-access-zh4th\") pod \"node-exporter-dsxrz\" (UID: \"df73b602-cb9a-4072-bd26-88bef578eef5\") " pod="openshift-monitoring/node-exporter-dsxrz" Sep 10 03:52:58.565564 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:58.565415 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/df73b602-cb9a-4072-bd26-88bef578eef5-node-exporter-accelerators-collector-config\") pod \"node-exporter-dsxrz\" (UID: \"df73b602-cb9a-4072-bd26-88bef578eef5\") " pod="openshift-monitoring/node-exporter-dsxrz" Sep 10 03:52:58.565564 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:58.565442 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/df73b602-cb9a-4072-bd26-88bef578eef5-metrics-client-ca\") pod \"node-exporter-dsxrz\" (UID: \"df73b602-cb9a-4072-bd26-88bef578eef5\") " pod="openshift-monitoring/node-exporter-dsxrz" Sep 10 03:52:58.565564 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:58.565468 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/df73b602-cb9a-4072-bd26-88bef578eef5-root\") pod \"node-exporter-dsxrz\" (UID: \"df73b602-cb9a-4072-bd26-88bef578eef5\") " pod="openshift-monitoring/node-exporter-dsxrz" Sep 10 03:52:58.565564 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:58.565504 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/df73b602-cb9a-4072-bd26-88bef578eef5-sys\") pod \"node-exporter-dsxrz\" (UID: \"df73b602-cb9a-4072-bd26-88bef578eef5\") " pod="openshift-monitoring/node-exporter-dsxrz" Sep 10 03:52:58.565564 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:58.565536 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/df73b602-cb9a-4072-bd26-88bef578eef5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dsxrz\" (UID: \"df73b602-cb9a-4072-bd26-88bef578eef5\") " pod="openshift-monitoring/node-exporter-dsxrz" Sep 10 03:52:58.565811 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:58.565573 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/df73b602-cb9a-4072-bd26-88bef578eef5-root\") pod \"node-exporter-dsxrz\" (UID: \"df73b602-cb9a-4072-bd26-88bef578eef5\") " pod="openshift-monitoring/node-exporter-dsxrz" Sep 10 03:52:58.565811 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:58.565581 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/df73b602-cb9a-4072-bd26-88bef578eef5-node-exporter-tls\") pod \"node-exporter-dsxrz\" (UID: \"df73b602-cb9a-4072-bd26-88bef578eef5\") " pod="openshift-monitoring/node-exporter-dsxrz" Sep 10 03:52:58.565811 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:58.565588 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/df73b602-cb9a-4072-bd26-88bef578eef5-sys\") pod \"node-exporter-dsxrz\" (UID: \"df73b602-cb9a-4072-bd26-88bef578eef5\") " pod="openshift-monitoring/node-exporter-dsxrz" Sep 10 03:52:58.565811 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:58.565652 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/df73b602-cb9a-4072-bd26-88bef578eef5-node-exporter-wtmp\") pod \"node-exporter-dsxrz\" (UID: \"df73b602-cb9a-4072-bd26-88bef578eef5\") " pod="openshift-monitoring/node-exporter-dsxrz" Sep 10 03:52:58.565811 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:52:58.565686 2581 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 10 03:52:58.565811 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:58.565690 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/df73b602-cb9a-4072-bd26-88bef578eef5-node-exporter-textfile\") pod \"node-exporter-dsxrz\" (UID: \"df73b602-cb9a-4072-bd26-88bef578eef5\") " pod="openshift-monitoring/node-exporter-dsxrz" Sep 10 03:52:58.565811 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:52:58.565740 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df73b602-cb9a-4072-bd26-88bef578eef5-node-exporter-tls podName:df73b602-cb9a-4072-bd26-88bef578eef5 nodeName:}" failed. No retries permitted until 2026-09-10 03:52:59.065722483 +0000 UTC m=+292.591101583 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/df73b602-cb9a-4072-bd26-88bef578eef5-node-exporter-tls") pod "node-exporter-dsxrz" (UID: "df73b602-cb9a-4072-bd26-88bef578eef5") : secret "node-exporter-tls" not found Sep 10 03:52:58.566106 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:58.565835 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/df73b602-cb9a-4072-bd26-88bef578eef5-node-exporter-wtmp\") pod \"node-exporter-dsxrz\" (UID: \"df73b602-cb9a-4072-bd26-88bef578eef5\") " pod="openshift-monitoring/node-exporter-dsxrz" Sep 10 03:52:58.566106 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:58.566049 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/df73b602-cb9a-4072-bd26-88bef578eef5-metrics-client-ca\") pod \"node-exporter-dsxrz\" (UID: \"df73b602-cb9a-4072-bd26-88bef578eef5\") " pod="openshift-monitoring/node-exporter-dsxrz" Sep 10 03:52:58.566106 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:58.566059 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/df73b602-cb9a-4072-bd26-88bef578eef5-node-exporter-textfile\") pod \"node-exporter-dsxrz\" (UID: \"df73b602-cb9a-4072-bd26-88bef578eef5\") " pod="openshift-monitoring/node-exporter-dsxrz" Sep 10 03:52:58.566106 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:58.566070 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/df73b602-cb9a-4072-bd26-88bef578eef5-node-exporter-accelerators-collector-config\") pod \"node-exporter-dsxrz\" (UID: \"df73b602-cb9a-4072-bd26-88bef578eef5\") " pod="openshift-monitoring/node-exporter-dsxrz" Sep 10 03:52:58.567897 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:58.567880 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/df73b602-cb9a-4072-bd26-88bef578eef5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dsxrz\" (UID: \"df73b602-cb9a-4072-bd26-88bef578eef5\") " pod="openshift-monitoring/node-exporter-dsxrz" Sep 10 03:52:58.573457 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:58.573431 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zh4th\" (UniqueName: \"kubernetes.io/projected/df73b602-cb9a-4072-bd26-88bef578eef5-kube-api-access-zh4th\") pod \"node-exporter-dsxrz\" (UID: \"df73b602-cb9a-4072-bd26-88bef578eef5\") " pod="openshift-monitoring/node-exporter-dsxrz" Sep 10 03:52:59.070079 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:59.070047 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/df73b602-cb9a-4072-bd26-88bef578eef5-node-exporter-tls\") pod \"node-exporter-dsxrz\" (UID: \"df73b602-cb9a-4072-bd26-88bef578eef5\") " pod="openshift-monitoring/node-exporter-dsxrz" Sep 10 03:52:59.072446 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:59.072429 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/df73b602-cb9a-4072-bd26-88bef578eef5-node-exporter-tls\") pod \"node-exporter-dsxrz\" (UID: \"df73b602-cb9a-4072-bd26-88bef578eef5\") " pod="openshift-monitoring/node-exporter-dsxrz" Sep 10 03:52:59.262571 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:59.262524 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dsxrz" Sep 10 03:52:59.273037 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:52:59.273007 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf73b602_cb9a_4072_bd26_88bef578eef5.slice/crio-5a3cfebcefaa091a039655b67311bc11db27708e019824d4573c375121ed2708 WatchSource:0}: Error finding container 5a3cfebcefaa091a039655b67311bc11db27708e019824d4573c375121ed2708: Status 404 returned error can't find the container with id 5a3cfebcefaa091a039655b67311bc11db27708e019824d4573c375121ed2708 Sep 10 03:52:59.857891 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:52:59.857844 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dsxrz" event={"ID":"df73b602-cb9a-4072-bd26-88bef578eef5","Type":"ContainerStarted","Data":"5a3cfebcefaa091a039655b67311bc11db27708e019824d4573c375121ed2708"} Sep 10 03:53:00.862859 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:00.862829 2581 generic.go:356] "Generic (PLEG): container finished" podID="df73b602-cb9a-4072-bd26-88bef578eef5" containerID="2e5556b07c24eb10dc45a436611eb2cef1c0d90f5863759d91550bd73922af54" exitCode=0 Sep 10 03:53:00.863235 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:00.862876 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dsxrz" event={"ID":"df73b602-cb9a-4072-bd26-88bef578eef5","Type":"ContainerDied","Data":"2e5556b07c24eb10dc45a436611eb2cef1c0d90f5863759d91550bd73922af54"} Sep 10 03:53:01.200229 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:01.200152 2581 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d9676fc7b-6zjvl" podUID="005bbc1c-6e3d-4b66-b0db-4af80d7e525a" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Sep 10 03:53:01.200229 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:01.200218 2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d9676fc7b-6zjvl" Sep 10 03:53:01.200695 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:01.200665 2581 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"9e517dd9af5e7e4d21cf27224bf00e45ec196163e0b8cd6ef5cabb295b892683"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d9676fc7b-6zjvl" containerMessage="Container service-proxy failed liveness probe, will be restarted" Sep 10 03:53:01.200742 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:01.200715 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d9676fc7b-6zjvl" podUID="005bbc1c-6e3d-4b66-b0db-4af80d7e525a" containerName="service-proxy" containerID="cri-o://9e517dd9af5e7e4d21cf27224bf00e45ec196163e0b8cd6ef5cabb295b892683" gracePeriod=30 Sep 10 03:53:01.868657 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:01.868623 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dsxrz" event={"ID":"df73b602-cb9a-4072-bd26-88bef578eef5","Type":"ContainerStarted","Data":"aba13b7a4c183828f4bab37baccfb32b0f21c9e6e4a6e3868032958aa5e5d8f1"} Sep 10 03:53:01.869100 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:01.868678 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dsxrz" event={"ID":"df73b602-cb9a-4072-bd26-88bef578eef5","Type":"ContainerStarted","Data":"2fc0d42f352dc72473a0f1695556b7beb9d014242afd19b6c94489facef36576"} Sep 10 03:53:01.870944 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:01.870918 2581 generic.go:356] "Generic (PLEG): container finished" podID="005bbc1c-6e3d-4b66-b0db-4af80d7e525a" containerID="9e517dd9af5e7e4d21cf27224bf00e45ec196163e0b8cd6ef5cabb295b892683" exitCode=2 Sep 10 03:53:01.871061 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:01.871000 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d9676fc7b-6zjvl" event={"ID":"005bbc1c-6e3d-4b66-b0db-4af80d7e525a","Type":"ContainerDied","Data":"9e517dd9af5e7e4d21cf27224bf00e45ec196163e0b8cd6ef5cabb295b892683"} Sep 10 03:53:01.871061 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:01.871036 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d9676fc7b-6zjvl" event={"ID":"005bbc1c-6e3d-4b66-b0db-4af80d7e525a","Type":"ContainerStarted","Data":"95ed9a65a42b2c1810701825b1c23b273c2dc7ec1e0a7307532f67234e5d1746"} Sep 10 03:53:01.889671 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:01.889621 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-dsxrz" podStartSLOduration=2.741861512 podStartE2EDuration="3.889606647s" totalImagesPullingTime="1.078459205s" totalInitContainerRuntime="69.28593ms" isStatefulPod=true podCreationTimestamp="2026-09-10 03:52:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:53:01.888560689 +0000 UTC m=+295.413939776" watchObservedRunningTime="2026-09-10 03:53:01.889606647 +0000 UTC m=+295.414985752" Sep 10 03:53:01.938043 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:53:01.938003 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-htpt9" podUID="fca9ba6d-4934-47e4-8be5-4296c7a6c530" Sep 10 03:53:03.098605 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:03.098574 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-jc5tg"] Sep 10 03:53:03.101536 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:03.101520 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-jc5tg" Sep 10 03:53:03.103671 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:03.103648 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Sep 10 03:53:03.103781 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:03.103725 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-xgk9x\"" Sep 10 03:53:03.109332 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:03.109302 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-jc5tg"] Sep 10 03:53:03.201724 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:03.201684 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7a060837-9c83-47f5-8425-df493263a67d-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-jc5tg\" (UID: \"7a060837-9c83-47f5-8425-df493263a67d\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-jc5tg" Sep 10 03:53:03.201893 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:03.201743 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l578d\" (UniqueName: \"kubernetes.io/projected/7a060837-9c83-47f5-8425-df493263a67d-kube-api-access-l578d\") pod \"monitoring-plugin-85bbbbbf4-jc5tg\" (UID: \"7a060837-9c83-47f5-8425-df493263a67d\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-jc5tg" Sep 10 03:53:03.302682 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:03.302651 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7a060837-9c83-47f5-8425-df493263a67d-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-jc5tg\" (UID: \"7a060837-9c83-47f5-8425-df493263a67d\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-jc5tg" Sep 10 03:53:03.302826 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:03.302708 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l578d\" (UniqueName: \"kubernetes.io/projected/7a060837-9c83-47f5-8425-df493263a67d-kube-api-access-l578d\") pod \"monitoring-plugin-85bbbbbf4-jc5tg\" (UID: \"7a060837-9c83-47f5-8425-df493263a67d\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-jc5tg" Sep 10 03:53:03.305309 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:03.305280 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7a060837-9c83-47f5-8425-df493263a67d-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-jc5tg\" (UID: \"7a060837-9c83-47f5-8425-df493263a67d\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-jc5tg" Sep 10 03:53:03.314649 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:03.314622 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l578d\" (UniqueName: \"kubernetes.io/projected/7a060837-9c83-47f5-8425-df493263a67d-kube-api-access-l578d\") pod \"monitoring-plugin-85bbbbbf4-jc5tg\" (UID: \"7a060837-9c83-47f5-8425-df493263a67d\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-jc5tg" Sep 10 03:53:03.411362 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:03.411283 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-jc5tg" Sep 10 03:53:03.678187 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:03.678114 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-jc5tg"] Sep 10 03:53:03.681445 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:53:03.681414 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a060837_9c83_47f5_8425_df493263a67d.slice/crio-59442af845594decb2ab1bab6b44069b4d7a1656b80b3c26a1a69db120a38645 WatchSource:0}: Error finding container 59442af845594decb2ab1bab6b44069b4d7a1656b80b3c26a1a69db120a38645: Status 404 returned error can't find the container with id 59442af845594decb2ab1bab6b44069b4d7a1656b80b3c26a1a69db120a38645 Sep 10 03:53:03.840322 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:03.840295 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-kzds7" Sep 10 03:53:03.880244 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:03.880213 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-jc5tg" event={"ID":"7a060837-9c83-47f5-8425-df493263a67d","Type":"ContainerStarted","Data":"59442af845594decb2ab1bab6b44069b4d7a1656b80b3c26a1a69db120a38645"} Sep 10 03:53:05.888919 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:05.888878 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-jc5tg" event={"ID":"7a060837-9c83-47f5-8425-df493263a67d","Type":"ContainerStarted","Data":"363a65e4cc8c95055a714821ea44311332bfdb781fb619cc774204c55a02e976"} Sep 10 03:53:05.889355 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:05.889039 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-jc5tg" Sep 10 03:53:05.893778 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:05.893745 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-jc5tg" Sep 10 03:53:05.903617 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:05.903576 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-jc5tg" podStartSLOduration=1.512380953 podStartE2EDuration="2.903561709s" totalImagesPullingTime="1.391180756s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:53:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:53:05.90331833 +0000 UTC m=+299.428697437" watchObservedRunningTime="2026-09-10 03:53:05.903561709 +0000 UTC m=+299.428940814" Sep 10 03:53:06.845607 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:06.845578 2581 kubelet.go:1648] "Image garbage collection succeeded" Sep 10 03:53:15.936987 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:15.936935 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htpt9" Sep 10 03:53:19.115997 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:19.115951 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7449df8cbc-tgqpq"] Sep 10 03:53:19.119753 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:19.119730 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7449df8cbc-tgqpq" Sep 10 03:53:19.122317 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:19.122294 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 10 03:53:19.122419 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:19.122295 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 10 03:53:19.123238 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:19.123215 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 10 03:53:19.123346 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:19.123249 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 10 03:53:19.123346 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:19.123215 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 10 03:53:19.123566 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:19.123551 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 10 03:53:19.123646 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:19.123601 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-hrv88\"" Sep 10 03:53:19.123646 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:19.123606 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 10 03:53:19.166385 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:19.166352 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7449df8cbc-tgqpq"] Sep 10 03:53:19.223725 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:19.223693 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9bf29e82-196a-4a5b-9504-f4b245b026e9-console-oauth-config\") pod \"console-7449df8cbc-tgqpq\" (UID: \"9bf29e82-196a-4a5b-9504-f4b245b026e9\") " pod="openshift-console/console-7449df8cbc-tgqpq" Sep 10 03:53:19.223725 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:19.223726 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9qt2\" (UniqueName: \"kubernetes.io/projected/9bf29e82-196a-4a5b-9504-f4b245b026e9-kube-api-access-h9qt2\") pod \"console-7449df8cbc-tgqpq\" (UID: \"9bf29e82-196a-4a5b-9504-f4b245b026e9\") " pod="openshift-console/console-7449df8cbc-tgqpq" Sep 10 03:53:19.223949 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:19.223750 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9bf29e82-196a-4a5b-9504-f4b245b026e9-service-ca\") pod \"console-7449df8cbc-tgqpq\" (UID: \"9bf29e82-196a-4a5b-9504-f4b245b026e9\") " pod="openshift-console/console-7449df8cbc-tgqpq" Sep 10 03:53:19.223949 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:19.223860 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9bf29e82-196a-4a5b-9504-f4b245b026e9-oauth-serving-cert\") pod \"console-7449df8cbc-tgqpq\" (UID: \"9bf29e82-196a-4a5b-9504-f4b245b026e9\") " pod="openshift-console/console-7449df8cbc-tgqpq" Sep 10 03:53:19.223949 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:19.223898 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9bf29e82-196a-4a5b-9504-f4b245b026e9-console-serving-cert\") pod \"console-7449df8cbc-tgqpq\" (UID: \"9bf29e82-196a-4a5b-9504-f4b245b026e9\") " pod="openshift-console/console-7449df8cbc-tgqpq" Sep 10 03:53:19.223949 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:19.223936 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9bf29e82-196a-4a5b-9504-f4b245b026e9-console-config\") pod \"console-7449df8cbc-tgqpq\" (UID: \"9bf29e82-196a-4a5b-9504-f4b245b026e9\") " pod="openshift-console/console-7449df8cbc-tgqpq" Sep 10 03:53:19.324860 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:19.324824 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9bf29e82-196a-4a5b-9504-f4b245b026e9-console-serving-cert\") pod \"console-7449df8cbc-tgqpq\" (UID: \"9bf29e82-196a-4a5b-9504-f4b245b026e9\") " pod="openshift-console/console-7449df8cbc-tgqpq" Sep 10 03:53:19.325032 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:19.324867 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9bf29e82-196a-4a5b-9504-f4b245b026e9-console-config\") pod \"console-7449df8cbc-tgqpq\" (UID: \"9bf29e82-196a-4a5b-9504-f4b245b026e9\") " pod="openshift-console/console-7449df8cbc-tgqpq" Sep 10 03:53:19.325032 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:19.324925 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9bf29e82-196a-4a5b-9504-f4b245b026e9-console-oauth-config\") pod \"console-7449df8cbc-tgqpq\" (UID: \"9bf29e82-196a-4a5b-9504-f4b245b026e9\") " pod="openshift-console/console-7449df8cbc-tgqpq" Sep 10 03:53:19.325032 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:19.324947 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9qt2\" (UniqueName: \"kubernetes.io/projected/9bf29e82-196a-4a5b-9504-f4b245b026e9-kube-api-access-h9qt2\") pod \"console-7449df8cbc-tgqpq\" (UID: \"9bf29e82-196a-4a5b-9504-f4b245b026e9\") " pod="openshift-console/console-7449df8cbc-tgqpq" Sep 10 03:53:19.325032 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:19.324996 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9bf29e82-196a-4a5b-9504-f4b245b026e9-service-ca\") pod \"console-7449df8cbc-tgqpq\" (UID: \"9bf29e82-196a-4a5b-9504-f4b245b026e9\") " pod="openshift-console/console-7449df8cbc-tgqpq" Sep 10 03:53:19.325223 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:19.325041 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9bf29e82-196a-4a5b-9504-f4b245b026e9-oauth-serving-cert\") pod \"console-7449df8cbc-tgqpq\" (UID: \"9bf29e82-196a-4a5b-9504-f4b245b026e9\") " pod="openshift-console/console-7449df8cbc-tgqpq" Sep 10 03:53:19.325737 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:19.325717 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9bf29e82-196a-4a5b-9504-f4b245b026e9-oauth-serving-cert\") pod \"console-7449df8cbc-tgqpq\" (UID: \"9bf29e82-196a-4a5b-9504-f4b245b026e9\") " pod="openshift-console/console-7449df8cbc-tgqpq" Sep 10 03:53:19.325837 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:19.325734 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9bf29e82-196a-4a5b-9504-f4b245b026e9-service-ca\") pod \"console-7449df8cbc-tgqpq\" (UID: \"9bf29e82-196a-4a5b-9504-f4b245b026e9\") " pod="openshift-console/console-7449df8cbc-tgqpq" Sep 10 03:53:19.325949 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:19.325929 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9bf29e82-196a-4a5b-9504-f4b245b026e9-console-config\") pod \"console-7449df8cbc-tgqpq\" (UID: \"9bf29e82-196a-4a5b-9504-f4b245b026e9\") " pod="openshift-console/console-7449df8cbc-tgqpq" Sep 10 03:53:19.327416 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:19.327391 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9bf29e82-196a-4a5b-9504-f4b245b026e9-console-oauth-config\") pod \"console-7449df8cbc-tgqpq\" (UID: \"9bf29e82-196a-4a5b-9504-f4b245b026e9\") " pod="openshift-console/console-7449df8cbc-tgqpq" Sep 10 03:53:19.327491 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:19.327458 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9bf29e82-196a-4a5b-9504-f4b245b026e9-console-serving-cert\") pod \"console-7449df8cbc-tgqpq\" (UID: \"9bf29e82-196a-4a5b-9504-f4b245b026e9\") " pod="openshift-console/console-7449df8cbc-tgqpq" Sep 10 03:53:19.334712 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:19.334692 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9qt2\" (UniqueName: \"kubernetes.io/projected/9bf29e82-196a-4a5b-9504-f4b245b026e9-kube-api-access-h9qt2\") pod \"console-7449df8cbc-tgqpq\" (UID: \"9bf29e82-196a-4a5b-9504-f4b245b026e9\") " pod="openshift-console/console-7449df8cbc-tgqpq" Sep 10 03:53:19.428451 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:19.428360 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7449df8cbc-tgqpq" Sep 10 03:53:19.697747 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:19.697674 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7449df8cbc-tgqpq"] Sep 10 03:53:19.700680 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:53:19.700658 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bf29e82_196a_4a5b_9504_f4b245b026e9.slice/crio-0b053f79ac80ad5a4a0d61ce27044448268c8ba45a9d36d2b7b2abad703228d3 WatchSource:0}: Error finding container 0b053f79ac80ad5a4a0d61ce27044448268c8ba45a9d36d2b7b2abad703228d3: Status 404 returned error can't find the container with id 0b053f79ac80ad5a4a0d61ce27044448268c8ba45a9d36d2b7b2abad703228d3 Sep 10 03:53:19.702936 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:19.702915 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 03:53:19.934053 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:19.934019 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7449df8cbc-tgqpq" event={"ID":"9bf29e82-196a-4a5b-9504-f4b245b026e9","Type":"ContainerStarted","Data":"0b053f79ac80ad5a4a0d61ce27044448268c8ba45a9d36d2b7b2abad703228d3"} Sep 10 03:53:23.952043 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:23.952007 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7449df8cbc-tgqpq" event={"ID":"9bf29e82-196a-4a5b-9504-f4b245b026e9","Type":"ContainerStarted","Data":"dbc84356cc8b45dcfc01c9797156adf61ad4e81a474e80a7c1c5d60ab38a5986"} Sep 10 03:53:23.969113 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:23.969072 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7449df8cbc-tgqpq" podStartSLOduration=1.478344874 podStartE2EDuration="4.969056974s" totalImagesPullingTime="3.4907121s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:53:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:53:23.967856637 +0000 UTC m=+317.493235741" watchObservedRunningTime="2026-09-10 03:53:23.969056974 +0000 UTC m=+317.494436079" Sep 10 03:53:26.536268 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:26.536239 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6994587fdc-wrtxz"] Sep 10 03:53:26.538258 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:26.538243 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6994587fdc-wrtxz" Sep 10 03:53:26.546691 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:26.546667 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 10 03:53:26.547788 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:26.547766 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6994587fdc-wrtxz"] Sep 10 03:53:26.689088 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:26.689039 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3fdd11d1-47a9-4727-92b9-7d035e58b8ed-oauth-serving-cert\") pod \"console-6994587fdc-wrtxz\" (UID: \"3fdd11d1-47a9-4727-92b9-7d035e58b8ed\") " pod="openshift-console/console-6994587fdc-wrtxz" Sep 10 03:53:26.689088 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:26.689092 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3fdd11d1-47a9-4727-92b9-7d035e58b8ed-console-config\") pod \"console-6994587fdc-wrtxz\" (UID: \"3fdd11d1-47a9-4727-92b9-7d035e58b8ed\") " pod="openshift-console/console-6994587fdc-wrtxz" Sep 10 03:53:26.689315 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:26.689176 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3fdd11d1-47a9-4727-92b9-7d035e58b8ed-console-serving-cert\") pod \"console-6994587fdc-wrtxz\" (UID: \"3fdd11d1-47a9-4727-92b9-7d035e58b8ed\") " pod="openshift-console/console-6994587fdc-wrtxz" Sep 10 03:53:26.689315 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:26.689243 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3fdd11d1-47a9-4727-92b9-7d035e58b8ed-service-ca\") pod \"console-6994587fdc-wrtxz\" (UID: \"3fdd11d1-47a9-4727-92b9-7d035e58b8ed\") " pod="openshift-console/console-6994587fdc-wrtxz" Sep 10 03:53:26.689315 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:26.689265 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcwsd\" (UniqueName: \"kubernetes.io/projected/3fdd11d1-47a9-4727-92b9-7d035e58b8ed-kube-api-access-lcwsd\") pod \"console-6994587fdc-wrtxz\" (UID: \"3fdd11d1-47a9-4727-92b9-7d035e58b8ed\") " pod="openshift-console/console-6994587fdc-wrtxz" Sep 10 03:53:26.689315 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:26.689281 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fdd11d1-47a9-4727-92b9-7d035e58b8ed-trusted-ca-bundle\") pod \"console-6994587fdc-wrtxz\" (UID: \"3fdd11d1-47a9-4727-92b9-7d035e58b8ed\") " pod="openshift-console/console-6994587fdc-wrtxz" Sep 10 03:53:26.689315 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:26.689300 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3fdd11d1-47a9-4727-92b9-7d035e58b8ed-console-oauth-config\") pod \"console-6994587fdc-wrtxz\" (UID: \"3fdd11d1-47a9-4727-92b9-7d035e58b8ed\") " pod="openshift-console/console-6994587fdc-wrtxz" Sep 10 03:53:26.789917 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:26.789851 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fdd11d1-47a9-4727-92b9-7d035e58b8ed-trusted-ca-bundle\") pod \"console-6994587fdc-wrtxz\" (UID: \"3fdd11d1-47a9-4727-92b9-7d035e58b8ed\") " pod="openshift-console/console-6994587fdc-wrtxz" Sep 10 03:53:26.789917 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:26.789888 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3fdd11d1-47a9-4727-92b9-7d035e58b8ed-console-oauth-config\") pod \"console-6994587fdc-wrtxz\" (UID: \"3fdd11d1-47a9-4727-92b9-7d035e58b8ed\") " pod="openshift-console/console-6994587fdc-wrtxz" Sep 10 03:53:26.789917 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:26.789911 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3fdd11d1-47a9-4727-92b9-7d035e58b8ed-oauth-serving-cert\") pod \"console-6994587fdc-wrtxz\" (UID: \"3fdd11d1-47a9-4727-92b9-7d035e58b8ed\") " pod="openshift-console/console-6994587fdc-wrtxz" Sep 10 03:53:26.790145 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:26.789939 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3fdd11d1-47a9-4727-92b9-7d035e58b8ed-console-config\") pod \"console-6994587fdc-wrtxz\" (UID: \"3fdd11d1-47a9-4727-92b9-7d035e58b8ed\") " pod="openshift-console/console-6994587fdc-wrtxz" Sep 10 03:53:26.790145 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:26.790034 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3fdd11d1-47a9-4727-92b9-7d035e58b8ed-console-serving-cert\") pod \"console-6994587fdc-wrtxz\" (UID: \"3fdd11d1-47a9-4727-92b9-7d035e58b8ed\") " pod="openshift-console/console-6994587fdc-wrtxz" Sep 10 03:53:26.790145 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:26.790080 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3fdd11d1-47a9-4727-92b9-7d035e58b8ed-service-ca\") pod \"console-6994587fdc-wrtxz\" (UID: \"3fdd11d1-47a9-4727-92b9-7d035e58b8ed\") " pod="openshift-console/console-6994587fdc-wrtxz" Sep 10 03:53:26.790145 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:26.790104 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcwsd\" (UniqueName: \"kubernetes.io/projected/3fdd11d1-47a9-4727-92b9-7d035e58b8ed-kube-api-access-lcwsd\") pod \"console-6994587fdc-wrtxz\" (UID: \"3fdd11d1-47a9-4727-92b9-7d035e58b8ed\") " pod="openshift-console/console-6994587fdc-wrtxz" Sep 10 03:53:26.790696 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:26.790668 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3fdd11d1-47a9-4727-92b9-7d035e58b8ed-oauth-serving-cert\") pod \"console-6994587fdc-wrtxz\" (UID: \"3fdd11d1-47a9-4727-92b9-7d035e58b8ed\") " pod="openshift-console/console-6994587fdc-wrtxz" Sep 10 03:53:26.790696 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:26.790688 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3fdd11d1-47a9-4727-92b9-7d035e58b8ed-console-config\") pod \"console-6994587fdc-wrtxz\" (UID: \"3fdd11d1-47a9-4727-92b9-7d035e58b8ed\") " pod="openshift-console/console-6994587fdc-wrtxz" Sep 10 03:53:26.790866 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:26.790797 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fdd11d1-47a9-4727-92b9-7d035e58b8ed-trusted-ca-bundle\") pod \"console-6994587fdc-wrtxz\" (UID: \"3fdd11d1-47a9-4727-92b9-7d035e58b8ed\") " pod="openshift-console/console-6994587fdc-wrtxz" Sep 10 03:53:26.790866 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:26.790857 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3fdd11d1-47a9-4727-92b9-7d035e58b8ed-service-ca\") pod \"console-6994587fdc-wrtxz\" (UID: \"3fdd11d1-47a9-4727-92b9-7d035e58b8ed\") " pod="openshift-console/console-6994587fdc-wrtxz" Sep 10 03:53:26.792937 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:26.792913 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3fdd11d1-47a9-4727-92b9-7d035e58b8ed-console-oauth-config\") pod \"console-6994587fdc-wrtxz\" (UID: \"3fdd11d1-47a9-4727-92b9-7d035e58b8ed\") " pod="openshift-console/console-6994587fdc-wrtxz" Sep 10 03:53:26.793150 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:26.793129 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3fdd11d1-47a9-4727-92b9-7d035e58b8ed-console-serving-cert\") pod \"console-6994587fdc-wrtxz\" (UID: \"3fdd11d1-47a9-4727-92b9-7d035e58b8ed\") " pod="openshift-console/console-6994587fdc-wrtxz" Sep 10 03:53:26.798722 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:26.798699 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcwsd\" (UniqueName: \"kubernetes.io/projected/3fdd11d1-47a9-4727-92b9-7d035e58b8ed-kube-api-access-lcwsd\") pod \"console-6994587fdc-wrtxz\" (UID: \"3fdd11d1-47a9-4727-92b9-7d035e58b8ed\") " pod="openshift-console/console-6994587fdc-wrtxz" Sep 10 03:53:26.847771 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:26.847743 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6994587fdc-wrtxz" Sep 10 03:53:27.116776 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:27.116705 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6994587fdc-wrtxz"] Sep 10 03:53:27.119565 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:53:27.119544 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fdd11d1_47a9_4727_92b9_7d035e58b8ed.slice/crio-78a50de15fbebae73dd905aa076855088b18662ccab657149d00b067eef70e46 WatchSource:0}: Error finding container 78a50de15fbebae73dd905aa076855088b18662ccab657149d00b067eef70e46: Status 404 returned error can't find the container with id 78a50de15fbebae73dd905aa076855088b18662ccab657149d00b067eef70e46 Sep 10 03:53:27.966454 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:27.966417 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6994587fdc-wrtxz" event={"ID":"3fdd11d1-47a9-4727-92b9-7d035e58b8ed","Type":"ContainerStarted","Data":"588b584a82c65ef6925dbe381122f29a2060bad8a7d128daff549fcf42b4efc3"} Sep 10 03:53:27.966454 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:27.966462 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6994587fdc-wrtxz" event={"ID":"3fdd11d1-47a9-4727-92b9-7d035e58b8ed","Type":"ContainerStarted","Data":"78a50de15fbebae73dd905aa076855088b18662ccab657149d00b067eef70e46"} Sep 10 03:53:27.985149 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:27.985078 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6994587fdc-wrtxz" podStartSLOduration=1.985065445 podStartE2EDuration="1.985065445s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:53:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:53:27.984606652 +0000 UTC m=+321.509985748" watchObservedRunningTime="2026-09-10 03:53:27.985065445 +0000 UTC m=+321.510444549" Sep 10 03:53:29.428751 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:29.428715 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7449df8cbc-tgqpq" Sep 10 03:53:29.429155 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:29.428784 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7449df8cbc-tgqpq" Sep 10 03:53:29.433554 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:29.433534 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7449df8cbc-tgqpq" Sep 10 03:53:29.977171 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:29.977136 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7449df8cbc-tgqpq" Sep 10 03:53:36.848337 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:36.848291 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6994587fdc-wrtxz" Sep 10 03:53:36.848732 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:36.848357 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6994587fdc-wrtxz" Sep 10 03:53:36.852887 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:36.852856 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6994587fdc-wrtxz" Sep 10 03:53:36.998233 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:36.998203 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6994587fdc-wrtxz" Sep 10 03:53:37.041209 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:53:37.041175 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7449df8cbc-tgqpq"] Sep 10 03:54:02.065166 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:02.065074 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7449df8cbc-tgqpq" podUID="9bf29e82-196a-4a5b-9504-f4b245b026e9" containerName="console" containerID="cri-o://dbc84356cc8b45dcfc01c9797156adf61ad4e81a474e80a7c1c5d60ab38a5986" gracePeriod=15 Sep 10 03:54:02.297269 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:02.297248 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7449df8cbc-tgqpq" Sep 10 03:54:02.335921 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:02.335843 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9bf29e82-196a-4a5b-9504-f4b245b026e9-console-serving-cert\") pod \"9bf29e82-196a-4a5b-9504-f4b245b026e9\" (UID: \"9bf29e82-196a-4a5b-9504-f4b245b026e9\") " Sep 10 03:54:02.335921 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:02.335879 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9bf29e82-196a-4a5b-9504-f4b245b026e9-oauth-serving-cert\") pod \"9bf29e82-196a-4a5b-9504-f4b245b026e9\" (UID: \"9bf29e82-196a-4a5b-9504-f4b245b026e9\") " Sep 10 03:54:02.335921 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:02.335900 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9bf29e82-196a-4a5b-9504-f4b245b026e9-console-oauth-config\") pod \"9bf29e82-196a-4a5b-9504-f4b245b026e9\" (UID: \"9bf29e82-196a-4a5b-9504-f4b245b026e9\") " Sep 10 03:54:02.336168 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:02.335933 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h9qt2\" (UniqueName: \"kubernetes.io/projected/9bf29e82-196a-4a5b-9504-f4b245b026e9-kube-api-access-h9qt2\") pod \"9bf29e82-196a-4a5b-9504-f4b245b026e9\" (UID: \"9bf29e82-196a-4a5b-9504-f4b245b026e9\") " Sep 10 03:54:02.336168 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:02.336036 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9bf29e82-196a-4a5b-9504-f4b245b026e9-console-config\") pod \"9bf29e82-196a-4a5b-9504-f4b245b026e9\" (UID: \"9bf29e82-196a-4a5b-9504-f4b245b026e9\") " Sep 10 03:54:02.336380 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:02.336354 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9bf29e82-196a-4a5b-9504-f4b245b026e9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9bf29e82-196a-4a5b-9504-f4b245b026e9" (UID: "9bf29e82-196a-4a5b-9504-f4b245b026e9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 03:54:02.336460 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:02.336406 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9bf29e82-196a-4a5b-9504-f4b245b026e9-console-config" (OuterVolumeSpecName: "console-config") pod "9bf29e82-196a-4a5b-9504-f4b245b026e9" (UID: "9bf29e82-196a-4a5b-9504-f4b245b026e9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 03:54:02.338335 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:02.338306 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bf29e82-196a-4a5b-9504-f4b245b026e9-kube-api-access-h9qt2" (OuterVolumeSpecName: "kube-api-access-h9qt2") pod "9bf29e82-196a-4a5b-9504-f4b245b026e9" (UID: "9bf29e82-196a-4a5b-9504-f4b245b026e9"). InnerVolumeSpecName "kube-api-access-h9qt2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 03:54:02.338335 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:02.338321 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9bf29e82-196a-4a5b-9504-f4b245b026e9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9bf29e82-196a-4a5b-9504-f4b245b026e9" (UID: "9bf29e82-196a-4a5b-9504-f4b245b026e9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 03:54:02.338474 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:02.338404 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9bf29e82-196a-4a5b-9504-f4b245b026e9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9bf29e82-196a-4a5b-9504-f4b245b026e9" (UID: "9bf29e82-196a-4a5b-9504-f4b245b026e9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 03:54:02.436780 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:02.436728 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9bf29e82-196a-4a5b-9504-f4b245b026e9-service-ca\") pod \"9bf29e82-196a-4a5b-9504-f4b245b026e9\" (UID: \"9bf29e82-196a-4a5b-9504-f4b245b026e9\") " Sep 10 03:54:02.437003 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:02.436889 2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9bf29e82-196a-4a5b-9504-f4b245b026e9-console-oauth-config\") on node \"ip-10-0-142-77.ec2.internal\" DevicePath \"\"" Sep 10 03:54:02.437003 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:02.436900 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9qt2\" (UniqueName: \"kubernetes.io/projected/9bf29e82-196a-4a5b-9504-f4b245b026e9-kube-api-access-h9qt2\") on node \"ip-10-0-142-77.ec2.internal\" DevicePath \"\"" Sep 10 03:54:02.437003 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:02.436923 2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9bf29e82-196a-4a5b-9504-f4b245b026e9-console-config\") on node \"ip-10-0-142-77.ec2.internal\" DevicePath \"\"" Sep 10 03:54:02.437003 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:02.436950 2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9bf29e82-196a-4a5b-9504-f4b245b026e9-console-serving-cert\") on node \"ip-10-0-142-77.ec2.internal\" DevicePath \"\"" Sep 10 03:54:02.437003 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:02.436959 2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9bf29e82-196a-4a5b-9504-f4b245b026e9-oauth-serving-cert\") on node \"ip-10-0-142-77.ec2.internal\" DevicePath \"\"" Sep 10 03:54:02.437262 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:02.437175 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9bf29e82-196a-4a5b-9504-f4b245b026e9-service-ca" (OuterVolumeSpecName: "service-ca") pod "9bf29e82-196a-4a5b-9504-f4b245b026e9" (UID: "9bf29e82-196a-4a5b-9504-f4b245b026e9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 03:54:02.537285 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:02.537232 2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9bf29e82-196a-4a5b-9504-f4b245b026e9-service-ca\") on node \"ip-10-0-142-77.ec2.internal\" DevicePath \"\"" Sep 10 03:54:03.081908 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:03.081875 2581 generic.go:356] "Generic (PLEG): container finished" podID="9bf29e82-196a-4a5b-9504-f4b245b026e9" containerID="dbc84356cc8b45dcfc01c9797156adf61ad4e81a474e80a7c1c5d60ab38a5986" exitCode=0 Sep 10 03:54:03.082404 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:03.081915 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7449df8cbc-tgqpq" Sep 10 03:54:03.082404 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:03.081932 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7449df8cbc-tgqpq" event={"ID":"9bf29e82-196a-4a5b-9504-f4b245b026e9","Type":"ContainerDied","Data":"dbc84356cc8b45dcfc01c9797156adf61ad4e81a474e80a7c1c5d60ab38a5986"} Sep 10 03:54:03.082404 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:03.082010 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7449df8cbc-tgqpq" event={"ID":"9bf29e82-196a-4a5b-9504-f4b245b026e9","Type":"ContainerDied","Data":"0b053f79ac80ad5a4a0d61ce27044448268c8ba45a9d36d2b7b2abad703228d3"} Sep 10 03:54:03.082404 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:03.082043 2581 scope.go:117] "RemoveContainer" containerID="dbc84356cc8b45dcfc01c9797156adf61ad4e81a474e80a7c1c5d60ab38a5986" Sep 10 03:54:03.089932 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:03.089914 2581 scope.go:117] "RemoveContainer" containerID="dbc84356cc8b45dcfc01c9797156adf61ad4e81a474e80a7c1c5d60ab38a5986" Sep 10 03:54:03.090242 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:54:03.090224 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbc84356cc8b45dcfc01c9797156adf61ad4e81a474e80a7c1c5d60ab38a5986\": container with ID starting with dbc84356cc8b45dcfc01c9797156adf61ad4e81a474e80a7c1c5d60ab38a5986 not found: ID does not exist" containerID="dbc84356cc8b45dcfc01c9797156adf61ad4e81a474e80a7c1c5d60ab38a5986" Sep 10 03:54:03.090301 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:03.090248 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbc84356cc8b45dcfc01c9797156adf61ad4e81a474e80a7c1c5d60ab38a5986"} err="failed to get container status \"dbc84356cc8b45dcfc01c9797156adf61ad4e81a474e80a7c1c5d60ab38a5986\": rpc error: code = NotFound desc = could not find container \"dbc84356cc8b45dcfc01c9797156adf61ad4e81a474e80a7c1c5d60ab38a5986\": container with ID starting with dbc84356cc8b45dcfc01c9797156adf61ad4e81a474e80a7c1c5d60ab38a5986 not found: ID does not exist" Sep 10 03:54:03.097442 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:03.097423 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7449df8cbc-tgqpq"] Sep 10 03:54:03.100326 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:03.100303 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7449df8cbc-tgqpq"] Sep 10 03:54:04.941114 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:04.941081 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9bf29e82-196a-4a5b-9504-f4b245b026e9" path="/var/lib/kubelet/pods/9bf29e82-196a-4a5b-9504-f4b245b026e9/volumes" Sep 10 03:54:20.863340 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:20.863309 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fca9ba6d-4934-47e4-8be5-4296c7a6c530-metrics-certs\") pod \"network-metrics-daemon-htpt9\" (UID: \"fca9ba6d-4934-47e4-8be5-4296c7a6c530\") " pod="openshift-multus/network-metrics-daemon-htpt9" Sep 10 03:54:20.865692 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:20.865673 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fca9ba6d-4934-47e4-8be5-4296c7a6c530-metrics-certs\") pod \"network-metrics-daemon-htpt9\" (UID: \"fca9ba6d-4934-47e4-8be5-4296c7a6c530\") " pod="openshift-multus/network-metrics-daemon-htpt9" Sep 10 03:54:21.040288 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:21.040258 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7r79s\"" Sep 10 03:54:21.048211 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:21.048178 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-htpt9" Sep 10 03:54:21.316443 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:21.316350 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-htpt9"] Sep 10 03:54:21.319085 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:54:21.319061 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfca9ba6d_4934_47e4_8be5_4296c7a6c530.slice/crio-acb658302a9e314660c9e52dbdcd9216e95e10f59f550d88dec45841a106cea2 WatchSource:0}: Error finding container acb658302a9e314660c9e52dbdcd9216e95e10f59f550d88dec45841a106cea2: Status 404 returned error can't find the container with id acb658302a9e314660c9e52dbdcd9216e95e10f59f550d88dec45841a106cea2 Sep 10 03:54:22.152541 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:22.152497 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-htpt9" event={"ID":"fca9ba6d-4934-47e4-8be5-4296c7a6c530","Type":"ContainerStarted","Data":"acb658302a9e314660c9e52dbdcd9216e95e10f59f550d88dec45841a106cea2"} Sep 10 03:54:23.158306 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:23.158268 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-htpt9" event={"ID":"fca9ba6d-4934-47e4-8be5-4296c7a6c530","Type":"ContainerStarted","Data":"4db2cf48659581b5ec153ac93238c25e1ace607558fe07b2279854b4c9beb290"} Sep 10 03:54:23.158770 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:23.158320 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-htpt9" event={"ID":"fca9ba6d-4934-47e4-8be5-4296c7a6c530","Type":"ContainerStarted","Data":"c17b63c96e0c27388f843130fbc44e975d8ecddb27cc9a9d78f1820583f173e4"} Sep 10 03:54:23.175153 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:23.175111 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-htpt9" podStartSLOduration=375.383649309 podStartE2EDuration="6m16.175096318s" totalImagesPullingTime="791.447009ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:48:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:54:23.17423088 +0000 UTC m=+376.699609988" watchObservedRunningTime="2026-09-10 03:54:23.175096318 +0000 UTC m=+376.700475423" Sep 10 03:54:30.779942 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:30.779906 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-cdfb499fc-592gv"] Sep 10 03:54:30.780364 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:30.780201 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bf29e82-196a-4a5b-9504-f4b245b026e9" containerName="console" Sep 10 03:54:30.780364 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:30.780216 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bf29e82-196a-4a5b-9504-f4b245b026e9" containerName="console" Sep 10 03:54:30.780364 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:30.780263 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bf29e82-196a-4a5b-9504-f4b245b026e9" containerName="console" Sep 10 03:54:30.783117 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:30.783101 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cdfb499fc-592gv" Sep 10 03:54:30.794730 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:30.794708 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cdfb499fc-592gv"] Sep 10 03:54:30.830702 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:30.830669 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f28d5cf1-f8b6-4eb2-9b96-a74b97085b39-console-serving-cert\") pod \"console-cdfb499fc-592gv\" (UID: \"f28d5cf1-f8b6-4eb2-9b96-a74b97085b39\") " pod="openshift-console/console-cdfb499fc-592gv" Sep 10 03:54:30.830702 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:30.830701 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pbrd\" (UniqueName: \"kubernetes.io/projected/f28d5cf1-f8b6-4eb2-9b96-a74b97085b39-kube-api-access-9pbrd\") pod \"console-cdfb499fc-592gv\" (UID: \"f28d5cf1-f8b6-4eb2-9b96-a74b97085b39\") " pod="openshift-console/console-cdfb499fc-592gv" Sep 10 03:54:30.830883 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:30.830719 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f28d5cf1-f8b6-4eb2-9b96-a74b97085b39-oauth-serving-cert\") pod \"console-cdfb499fc-592gv\" (UID: \"f28d5cf1-f8b6-4eb2-9b96-a74b97085b39\") " pod="openshift-console/console-cdfb499fc-592gv" Sep 10 03:54:30.830883 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:30.830744 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f28d5cf1-f8b6-4eb2-9b96-a74b97085b39-console-oauth-config\") pod \"console-cdfb499fc-592gv\" (UID: \"f28d5cf1-f8b6-4eb2-9b96-a74b97085b39\") " pod="openshift-console/console-cdfb499fc-592gv" Sep 10 03:54:30.830883 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:30.830822 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f28d5cf1-f8b6-4eb2-9b96-a74b97085b39-console-config\") pod \"console-cdfb499fc-592gv\" (UID: \"f28d5cf1-f8b6-4eb2-9b96-a74b97085b39\") " pod="openshift-console/console-cdfb499fc-592gv" Sep 10 03:54:30.830883 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:30.830854 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f28d5cf1-f8b6-4eb2-9b96-a74b97085b39-service-ca\") pod \"console-cdfb499fc-592gv\" (UID: \"f28d5cf1-f8b6-4eb2-9b96-a74b97085b39\") " pod="openshift-console/console-cdfb499fc-592gv" Sep 10 03:54:30.830883 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:30.830876 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f28d5cf1-f8b6-4eb2-9b96-a74b97085b39-trusted-ca-bundle\") pod \"console-cdfb499fc-592gv\" (UID: \"f28d5cf1-f8b6-4eb2-9b96-a74b97085b39\") " pod="openshift-console/console-cdfb499fc-592gv" Sep 10 03:54:30.931818 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:30.931786 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f28d5cf1-f8b6-4eb2-9b96-a74b97085b39-console-config\") pod \"console-cdfb499fc-592gv\" (UID: \"f28d5cf1-f8b6-4eb2-9b96-a74b97085b39\") " pod="openshift-console/console-cdfb499fc-592gv" Sep 10 03:54:30.932019 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:30.931826 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f28d5cf1-f8b6-4eb2-9b96-a74b97085b39-service-ca\") pod \"console-cdfb499fc-592gv\" (UID: \"f28d5cf1-f8b6-4eb2-9b96-a74b97085b39\") " pod="openshift-console/console-cdfb499fc-592gv" Sep 10 03:54:30.932019 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:30.931843 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f28d5cf1-f8b6-4eb2-9b96-a74b97085b39-trusted-ca-bundle\") pod \"console-cdfb499fc-592gv\" (UID: \"f28d5cf1-f8b6-4eb2-9b96-a74b97085b39\") " pod="openshift-console/console-cdfb499fc-592gv" Sep 10 03:54:30.932019 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:30.931864 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f28d5cf1-f8b6-4eb2-9b96-a74b97085b39-console-serving-cert\") pod \"console-cdfb499fc-592gv\" (UID: \"f28d5cf1-f8b6-4eb2-9b96-a74b97085b39\") " pod="openshift-console/console-cdfb499fc-592gv" Sep 10 03:54:30.932019 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:30.931898 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pbrd\" (UniqueName: \"kubernetes.io/projected/f28d5cf1-f8b6-4eb2-9b96-a74b97085b39-kube-api-access-9pbrd\") pod \"console-cdfb499fc-592gv\" (UID: \"f28d5cf1-f8b6-4eb2-9b96-a74b97085b39\") " pod="openshift-console/console-cdfb499fc-592gv" Sep 10 03:54:30.932019 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:30.931926 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f28d5cf1-f8b6-4eb2-9b96-a74b97085b39-oauth-serving-cert\") pod \"console-cdfb499fc-592gv\" (UID: \"f28d5cf1-f8b6-4eb2-9b96-a74b97085b39\") " pod="openshift-console/console-cdfb499fc-592gv" Sep 10 03:54:30.932019 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:30.931988 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f28d5cf1-f8b6-4eb2-9b96-a74b97085b39-console-oauth-config\") pod \"console-cdfb499fc-592gv\" (UID: \"f28d5cf1-f8b6-4eb2-9b96-a74b97085b39\") " pod="openshift-console/console-cdfb499fc-592gv" Sep 10 03:54:30.932628 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:30.932606 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f28d5cf1-f8b6-4eb2-9b96-a74b97085b39-service-ca\") pod \"console-cdfb499fc-592gv\" (UID: \"f28d5cf1-f8b6-4eb2-9b96-a74b97085b39\") " pod="openshift-console/console-cdfb499fc-592gv" Sep 10 03:54:30.932725 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:30.932667 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f28d5cf1-f8b6-4eb2-9b96-a74b97085b39-console-config\") pod \"console-cdfb499fc-592gv\" (UID: \"f28d5cf1-f8b6-4eb2-9b96-a74b97085b39\") " pod="openshift-console/console-cdfb499fc-592gv" Sep 10 03:54:30.932826 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:30.932800 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f28d5cf1-f8b6-4eb2-9b96-a74b97085b39-oauth-serving-cert\") pod \"console-cdfb499fc-592gv\" (UID: \"f28d5cf1-f8b6-4eb2-9b96-a74b97085b39\") " pod="openshift-console/console-cdfb499fc-592gv" Sep 10 03:54:30.933042 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:30.933021 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f28d5cf1-f8b6-4eb2-9b96-a74b97085b39-trusted-ca-bundle\") pod \"console-cdfb499fc-592gv\" (UID: \"f28d5cf1-f8b6-4eb2-9b96-a74b97085b39\") " pod="openshift-console/console-cdfb499fc-592gv" Sep 10 03:54:30.934411 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:30.934394 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f28d5cf1-f8b6-4eb2-9b96-a74b97085b39-console-oauth-config\") pod \"console-cdfb499fc-592gv\" (UID: \"f28d5cf1-f8b6-4eb2-9b96-a74b97085b39\") " pod="openshift-console/console-cdfb499fc-592gv" Sep 10 03:54:30.934649 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:30.934628 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f28d5cf1-f8b6-4eb2-9b96-a74b97085b39-console-serving-cert\") pod \"console-cdfb499fc-592gv\" (UID: \"f28d5cf1-f8b6-4eb2-9b96-a74b97085b39\") " pod="openshift-console/console-cdfb499fc-592gv" Sep 10 03:54:30.940222 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:30.940161 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pbrd\" (UniqueName: \"kubernetes.io/projected/f28d5cf1-f8b6-4eb2-9b96-a74b97085b39-kube-api-access-9pbrd\") pod \"console-cdfb499fc-592gv\" (UID: \"f28d5cf1-f8b6-4eb2-9b96-a74b97085b39\") " pod="openshift-console/console-cdfb499fc-592gv" Sep 10 03:54:31.091348 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:31.091263 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cdfb499fc-592gv" Sep 10 03:54:31.362149 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:31.362125 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cdfb499fc-592gv"] Sep 10 03:54:31.364732 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:54:31.364703 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf28d5cf1_f8b6_4eb2_9b96_a74b97085b39.slice/crio-30e370af756eaa5189eb1d4aa6976882c2a3c688465b28f18ff6edde702aa86e WatchSource:0}: Error finding container 30e370af756eaa5189eb1d4aa6976882c2a3c688465b28f18ff6edde702aa86e: Status 404 returned error can't find the container with id 30e370af756eaa5189eb1d4aa6976882c2a3c688465b28f18ff6edde702aa86e Sep 10 03:54:32.192077 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:32.192044 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cdfb499fc-592gv" event={"ID":"f28d5cf1-f8b6-4eb2-9b96-a74b97085b39","Type":"ContainerStarted","Data":"28318e8db41e1065828de89c28b2e5dc4542ee2c4670f276aaa5477e57d60e6f"} Sep 10 03:54:32.192535 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:32.192090 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cdfb499fc-592gv" event={"ID":"f28d5cf1-f8b6-4eb2-9b96-a74b97085b39","Type":"ContainerStarted","Data":"30e370af756eaa5189eb1d4aa6976882c2a3c688465b28f18ff6edde702aa86e"} Sep 10 03:54:32.209871 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:32.209831 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-cdfb499fc-592gv" podStartSLOduration=2.209817008 podStartE2EDuration="2.209817008s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:54:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:54:32.208302364 +0000 UTC m=+385.733681468" watchObservedRunningTime="2026-09-10 03:54:32.209817008 +0000 UTC m=+385.735196164" Sep 10 03:54:41.092112 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:41.092078 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-cdfb499fc-592gv" Sep 10 03:54:41.092485 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:41.092150 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-cdfb499fc-592gv" Sep 10 03:54:41.096573 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:41.096555 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-cdfb499fc-592gv" Sep 10 03:54:41.229313 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:41.229286 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-cdfb499fc-592gv" Sep 10 03:54:41.282502 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:54:41.282474 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6994587fdc-wrtxz"] Sep 10 03:55:06.302565 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:06.302508 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6994587fdc-wrtxz" podUID="3fdd11d1-47a9-4727-92b9-7d035e58b8ed" containerName="console" containerID="cri-o://588b584a82c65ef6925dbe381122f29a2060bad8a7d128daff549fcf42b4efc3" gracePeriod=15 Sep 10 03:55:06.541404 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:06.541378 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6994587fdc-wrtxz" Sep 10 03:55:06.685891 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:06.685802 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3fdd11d1-47a9-4727-92b9-7d035e58b8ed-console-config\") pod \"3fdd11d1-47a9-4727-92b9-7d035e58b8ed\" (UID: \"3fdd11d1-47a9-4727-92b9-7d035e58b8ed\") " Sep 10 03:55:06.685891 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:06.685845 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fdd11d1-47a9-4727-92b9-7d035e58b8ed-trusted-ca-bundle\") pod \"3fdd11d1-47a9-4727-92b9-7d035e58b8ed\" (UID: \"3fdd11d1-47a9-4727-92b9-7d035e58b8ed\") " Sep 10 03:55:06.685891 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:06.685874 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lcwsd\" (UniqueName: \"kubernetes.io/projected/3fdd11d1-47a9-4727-92b9-7d035e58b8ed-kube-api-access-lcwsd\") pod \"3fdd11d1-47a9-4727-92b9-7d035e58b8ed\" (UID: \"3fdd11d1-47a9-4727-92b9-7d035e58b8ed\") " Sep 10 03:55:06.685891 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:06.685893 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3fdd11d1-47a9-4727-92b9-7d035e58b8ed-console-oauth-config\") pod \"3fdd11d1-47a9-4727-92b9-7d035e58b8ed\" (UID: \"3fdd11d1-47a9-4727-92b9-7d035e58b8ed\") " Sep 10 03:55:06.686218 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:06.686034 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3fdd11d1-47a9-4727-92b9-7d035e58b8ed-service-ca\") pod \"3fdd11d1-47a9-4727-92b9-7d035e58b8ed\" (UID: \"3fdd11d1-47a9-4727-92b9-7d035e58b8ed\") " Sep 10 03:55:06.686218 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:06.686078 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3fdd11d1-47a9-4727-92b9-7d035e58b8ed-oauth-serving-cert\") pod \"3fdd11d1-47a9-4727-92b9-7d035e58b8ed\" (UID: \"3fdd11d1-47a9-4727-92b9-7d035e58b8ed\") " Sep 10 03:55:06.686218 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:06.686114 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3fdd11d1-47a9-4727-92b9-7d035e58b8ed-console-serving-cert\") pod \"3fdd11d1-47a9-4727-92b9-7d035e58b8ed\" (UID: \"3fdd11d1-47a9-4727-92b9-7d035e58b8ed\") " Sep 10 03:55:06.686375 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:06.686312 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fdd11d1-47a9-4727-92b9-7d035e58b8ed-console-config" (OuterVolumeSpecName: "console-config") pod "3fdd11d1-47a9-4727-92b9-7d035e58b8ed" (UID: "3fdd11d1-47a9-4727-92b9-7d035e58b8ed"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 03:55:06.686470 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:06.686439 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fdd11d1-47a9-4727-92b9-7d035e58b8ed-service-ca" (OuterVolumeSpecName: "service-ca") pod "3fdd11d1-47a9-4727-92b9-7d035e58b8ed" (UID: "3fdd11d1-47a9-4727-92b9-7d035e58b8ed"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 03:55:06.686522 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:06.686477 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fdd11d1-47a9-4727-92b9-7d035e58b8ed-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3fdd11d1-47a9-4727-92b9-7d035e58b8ed" (UID: "3fdd11d1-47a9-4727-92b9-7d035e58b8ed"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 03:55:06.686559 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:06.686520 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fdd11d1-47a9-4727-92b9-7d035e58b8ed-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3fdd11d1-47a9-4727-92b9-7d035e58b8ed" (UID: "3fdd11d1-47a9-4727-92b9-7d035e58b8ed"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 03:55:06.688274 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:06.688254 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fdd11d1-47a9-4727-92b9-7d035e58b8ed-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3fdd11d1-47a9-4727-92b9-7d035e58b8ed" (UID: "3fdd11d1-47a9-4727-92b9-7d035e58b8ed"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 03:55:06.688355 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:06.688268 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fdd11d1-47a9-4727-92b9-7d035e58b8ed-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3fdd11d1-47a9-4727-92b9-7d035e58b8ed" (UID: "3fdd11d1-47a9-4727-92b9-7d035e58b8ed"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 03:55:06.688394 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:06.688371 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fdd11d1-47a9-4727-92b9-7d035e58b8ed-kube-api-access-lcwsd" (OuterVolumeSpecName: "kube-api-access-lcwsd") pod "3fdd11d1-47a9-4727-92b9-7d035e58b8ed" (UID: "3fdd11d1-47a9-4727-92b9-7d035e58b8ed"). InnerVolumeSpecName "kube-api-access-lcwsd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 03:55:06.787282 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:06.787237 2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3fdd11d1-47a9-4727-92b9-7d035e58b8ed-console-config\") on node \"ip-10-0-142-77.ec2.internal\" DevicePath \"\"" Sep 10 03:55:06.787282 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:06.787277 2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fdd11d1-47a9-4727-92b9-7d035e58b8ed-trusted-ca-bundle\") on node \"ip-10-0-142-77.ec2.internal\" DevicePath \"\"" Sep 10 03:55:06.787282 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:06.787288 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lcwsd\" (UniqueName: \"kubernetes.io/projected/3fdd11d1-47a9-4727-92b9-7d035e58b8ed-kube-api-access-lcwsd\") on node \"ip-10-0-142-77.ec2.internal\" DevicePath \"\"" Sep 10 03:55:06.787282 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:06.787297 2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3fdd11d1-47a9-4727-92b9-7d035e58b8ed-console-oauth-config\") on node \"ip-10-0-142-77.ec2.internal\" DevicePath \"\"" Sep 10 03:55:06.787530 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:06.787305 2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3fdd11d1-47a9-4727-92b9-7d035e58b8ed-service-ca\") on node \"ip-10-0-142-77.ec2.internal\" DevicePath \"\"" Sep 10 03:55:06.787530 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:06.787314 2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3fdd11d1-47a9-4727-92b9-7d035e58b8ed-oauth-serving-cert\") on node \"ip-10-0-142-77.ec2.internal\" DevicePath \"\"" Sep 10 03:55:06.787530 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:06.787322 2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3fdd11d1-47a9-4727-92b9-7d035e58b8ed-console-serving-cert\") on node \"ip-10-0-142-77.ec2.internal\" DevicePath \"\"" Sep 10 03:55:07.316854 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:07.316822 2581 generic.go:356] "Generic (PLEG): container finished" podID="3fdd11d1-47a9-4727-92b9-7d035e58b8ed" containerID="588b584a82c65ef6925dbe381122f29a2060bad8a7d128daff549fcf42b4efc3" exitCode=0 Sep 10 03:55:07.317285 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:07.316884 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6994587fdc-wrtxz" Sep 10 03:55:07.317285 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:07.316883 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6994587fdc-wrtxz" event={"ID":"3fdd11d1-47a9-4727-92b9-7d035e58b8ed","Type":"ContainerDied","Data":"588b584a82c65ef6925dbe381122f29a2060bad8a7d128daff549fcf42b4efc3"} Sep 10 03:55:07.317285 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:07.317045 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6994587fdc-wrtxz" event={"ID":"3fdd11d1-47a9-4727-92b9-7d035e58b8ed","Type":"ContainerDied","Data":"78a50de15fbebae73dd905aa076855088b18662ccab657149d00b067eef70e46"} Sep 10 03:55:07.317285 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:07.317072 2581 scope.go:117] "RemoveContainer" containerID="588b584a82c65ef6925dbe381122f29a2060bad8a7d128daff549fcf42b4efc3" Sep 10 03:55:07.324714 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:07.324697 2581 scope.go:117] "RemoveContainer" containerID="588b584a82c65ef6925dbe381122f29a2060bad8a7d128daff549fcf42b4efc3" Sep 10 03:55:07.325029 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:55:07.325003 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"588b584a82c65ef6925dbe381122f29a2060bad8a7d128daff549fcf42b4efc3\": container with ID starting with 588b584a82c65ef6925dbe381122f29a2060bad8a7d128daff549fcf42b4efc3 not found: ID does not exist" containerID="588b584a82c65ef6925dbe381122f29a2060bad8a7d128daff549fcf42b4efc3" Sep 10 03:55:07.325129 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:07.325033 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"588b584a82c65ef6925dbe381122f29a2060bad8a7d128daff549fcf42b4efc3"} err="failed to get container status \"588b584a82c65ef6925dbe381122f29a2060bad8a7d128daff549fcf42b4efc3\": rpc error: code = NotFound desc = could not find container \"588b584a82c65ef6925dbe381122f29a2060bad8a7d128daff549fcf42b4efc3\": container with ID starting with 588b584a82c65ef6925dbe381122f29a2060bad8a7d128daff549fcf42b4efc3 not found: ID does not exist" Sep 10 03:55:07.332154 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:07.332135 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6994587fdc-wrtxz"] Sep 10 03:55:07.336031 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:07.336014 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6994587fdc-wrtxz"] Sep 10 03:55:08.940869 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:08.940830 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3fdd11d1-47a9-4727-92b9-7d035e58b8ed" path="/var/lib/kubelet/pods/3fdd11d1-47a9-4727-92b9-7d035e58b8ed/volumes" Sep 10 03:55:59.505350 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:59.505317 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-dpzv9"] Sep 10 03:55:59.505809 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:59.505542 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fdd11d1-47a9-4727-92b9-7d035e58b8ed" containerName="console" Sep 10 03:55:59.505809 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:59.505553 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fdd11d1-47a9-4727-92b9-7d035e58b8ed" containerName="console" Sep 10 03:55:59.505809 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:59.505610 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fdd11d1-47a9-4727-92b9-7d035e58b8ed" containerName="console" Sep 10 03:55:59.508301 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:59.508285 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-dpzv9" Sep 10 03:55:59.511950 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:59.511929 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Sep 10 03:55:59.512521 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:59.512506 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Sep 10 03:55:59.522387 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:59.522361 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-dpzv9"] Sep 10 03:55:59.560284 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:59.560254 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dfe96441-4657-4421-b632-152ae3327ecd-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-dpzv9\" (UID: \"dfe96441-4657-4421-b632-152ae3327ecd\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dpzv9" Sep 10 03:55:59.560433 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:59.560290 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nphn6\" (UniqueName: \"kubernetes.io/projected/dfe96441-4657-4421-b632-152ae3327ecd-kube-api-access-nphn6\") pod \"cert-manager-webhook-6486ffd997-dpzv9\" (UID: \"dfe96441-4657-4421-b632-152ae3327ecd\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dpzv9" Sep 10 03:55:59.660916 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:59.660881 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dfe96441-4657-4421-b632-152ae3327ecd-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-dpzv9\" (UID: \"dfe96441-4657-4421-b632-152ae3327ecd\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dpzv9" Sep 10 03:55:59.660916 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:59.660918 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nphn6\" (UniqueName: \"kubernetes.io/projected/dfe96441-4657-4421-b632-152ae3327ecd-kube-api-access-nphn6\") pod \"cert-manager-webhook-6486ffd997-dpzv9\" (UID: \"dfe96441-4657-4421-b632-152ae3327ecd\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dpzv9" Sep 10 03:55:59.675200 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:59.675175 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nphn6\" (UniqueName: \"kubernetes.io/projected/dfe96441-4657-4421-b632-152ae3327ecd-kube-api-access-nphn6\") pod \"cert-manager-webhook-6486ffd997-dpzv9\" (UID: \"dfe96441-4657-4421-b632-152ae3327ecd\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dpzv9" Sep 10 03:55:59.677328 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:59.677312 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dfe96441-4657-4421-b632-152ae3327ecd-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-dpzv9\" (UID: \"dfe96441-4657-4421-b632-152ae3327ecd\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dpzv9" Sep 10 03:55:59.816779 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:55:59.816749 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-dpzv9" Sep 10 03:56:00.083567 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:00.083520 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-dpzv9"] Sep 10 03:56:00.086221 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:56:00.086195 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddfe96441_4657_4421_b632_152ae3327ecd.slice/crio-8a161a58a8c6fba88e229373bb5177d487c91b2db38c6ce289822fe21175b5b3 WatchSource:0}: Error finding container 8a161a58a8c6fba88e229373bb5177d487c91b2db38c6ce289822fe21175b5b3: Status 404 returned error can't find the container with id 8a161a58a8c6fba88e229373bb5177d487c91b2db38c6ce289822fe21175b5b3 Sep 10 03:56:00.500119 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:00.500033 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-dpzv9" event={"ID":"dfe96441-4657-4421-b632-152ae3327ecd","Type":"ContainerStarted","Data":"8a161a58a8c6fba88e229373bb5177d487c91b2db38c6ce289822fe21175b5b3"} Sep 10 03:56:03.513903 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:03.513862 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-dpzv9" event={"ID":"dfe96441-4657-4421-b632-152ae3327ecd","Type":"ContainerStarted","Data":"2804a9b8c30f466198a832ede36ff5b8c0487a4585aa02184c25da8c56153a7e"} Sep 10 03:56:03.514309 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:03.514029 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-dpzv9" Sep 10 03:56:03.528779 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:03.528732 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-dpzv9" podStartSLOduration=1.254492538 podStartE2EDuration="4.528711613s" totalImagesPullingTime="3.274219075s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:55:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:56:03.528531088 +0000 UTC m=+477.053910194" watchObservedRunningTime="2026-09-10 03:56:03.528711613 +0000 UTC m=+477.054090719" Sep 10 03:56:09.519302 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:09.519267 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-dpzv9" Sep 10 03:56:33.914577 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:33.914543 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-797557847b-5kp5j"] Sep 10 03:56:33.917717 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:33.917699 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-797557847b-5kp5j" Sep 10 03:56:33.920217 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:33.920192 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 10 03:56:33.920336 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:33.920238 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 10 03:56:33.920393 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:33.920345 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 10 03:56:33.920524 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:33.920509 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-jgxxc\"" Sep 10 03:56:33.920565 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:33.920535 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 10 03:56:33.934351 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:33.934331 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-797557847b-5kp5j"] Sep 10 03:56:33.995273 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:33.995249 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xcw8\" (UniqueName: \"kubernetes.io/projected/8a0e0403-cfc8-485f-acf8-c56389c13825-kube-api-access-2xcw8\") pod \"opendatahub-operator-controller-manager-797557847b-5kp5j\" (UID: \"8a0e0403-cfc8-485f-acf8-c56389c13825\") " pod="opendatahub/opendatahub-operator-controller-manager-797557847b-5kp5j" Sep 10 03:56:33.995405 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:33.995282 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a0e0403-cfc8-485f-acf8-c56389c13825-tmp\") pod \"opendatahub-operator-controller-manager-797557847b-5kp5j\" (UID: \"8a0e0403-cfc8-485f-acf8-c56389c13825\") " pod="opendatahub/opendatahub-operator-controller-manager-797557847b-5kp5j" Sep 10 03:56:33.995405 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:33.995306 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8a0e0403-cfc8-485f-acf8-c56389c13825-apiservice-cert\") pod \"opendatahub-operator-controller-manager-797557847b-5kp5j\" (UID: \"8a0e0403-cfc8-485f-acf8-c56389c13825\") " pod="opendatahub/opendatahub-operator-controller-manager-797557847b-5kp5j" Sep 10 03:56:33.995405 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:33.995328 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8a0e0403-cfc8-485f-acf8-c56389c13825-webhook-cert\") pod \"opendatahub-operator-controller-manager-797557847b-5kp5j\" (UID: \"8a0e0403-cfc8-485f-acf8-c56389c13825\") " pod="opendatahub/opendatahub-operator-controller-manager-797557847b-5kp5j" Sep 10 03:56:33.995405 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:33.995382 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8a0e0403-cfc8-485f-acf8-c56389c13825-manifests\") pod \"opendatahub-operator-controller-manager-797557847b-5kp5j\" (UID: \"8a0e0403-cfc8-485f-acf8-c56389c13825\") " pod="opendatahub/opendatahub-operator-controller-manager-797557847b-5kp5j" Sep 10 03:56:34.096172 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:34.096145 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a0e0403-cfc8-485f-acf8-c56389c13825-tmp\") pod \"opendatahub-operator-controller-manager-797557847b-5kp5j\" (UID: \"8a0e0403-cfc8-485f-acf8-c56389c13825\") " pod="opendatahub/opendatahub-operator-controller-manager-797557847b-5kp5j" Sep 10 03:56:34.096306 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:34.096189 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8a0e0403-cfc8-485f-acf8-c56389c13825-apiservice-cert\") pod \"opendatahub-operator-controller-manager-797557847b-5kp5j\" (UID: \"8a0e0403-cfc8-485f-acf8-c56389c13825\") " pod="opendatahub/opendatahub-operator-controller-manager-797557847b-5kp5j" Sep 10 03:56:34.096306 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:34.096206 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8a0e0403-cfc8-485f-acf8-c56389c13825-webhook-cert\") pod \"opendatahub-operator-controller-manager-797557847b-5kp5j\" (UID: \"8a0e0403-cfc8-485f-acf8-c56389c13825\") " pod="opendatahub/opendatahub-operator-controller-manager-797557847b-5kp5j" Sep 10 03:56:34.096306 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:34.096237 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8a0e0403-cfc8-485f-acf8-c56389c13825-manifests\") pod \"opendatahub-operator-controller-manager-797557847b-5kp5j\" (UID: \"8a0e0403-cfc8-485f-acf8-c56389c13825\") " pod="opendatahub/opendatahub-operator-controller-manager-797557847b-5kp5j" Sep 10 03:56:34.096306 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:34.096277 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xcw8\" (UniqueName: \"kubernetes.io/projected/8a0e0403-cfc8-485f-acf8-c56389c13825-kube-api-access-2xcw8\") pod \"opendatahub-operator-controller-manager-797557847b-5kp5j\" (UID: \"8a0e0403-cfc8-485f-acf8-c56389c13825\") " pod="opendatahub/opendatahub-operator-controller-manager-797557847b-5kp5j" Sep 10 03:56:34.096613 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:34.096587 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8a0e0403-cfc8-485f-acf8-c56389c13825-manifests\") pod \"opendatahub-operator-controller-manager-797557847b-5kp5j\" (UID: \"8a0e0403-cfc8-485f-acf8-c56389c13825\") " pod="opendatahub/opendatahub-operator-controller-manager-797557847b-5kp5j" Sep 10 03:56:34.098487 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:34.098468 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a0e0403-cfc8-485f-acf8-c56389c13825-tmp\") pod \"opendatahub-operator-controller-manager-797557847b-5kp5j\" (UID: \"8a0e0403-cfc8-485f-acf8-c56389c13825\") " pod="opendatahub/opendatahub-operator-controller-manager-797557847b-5kp5j" Sep 10 03:56:34.098612 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:34.098595 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8a0e0403-cfc8-485f-acf8-c56389c13825-apiservice-cert\") pod \"opendatahub-operator-controller-manager-797557847b-5kp5j\" (UID: \"8a0e0403-cfc8-485f-acf8-c56389c13825\") " pod="opendatahub/opendatahub-operator-controller-manager-797557847b-5kp5j" Sep 10 03:56:34.098652 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:34.098631 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8a0e0403-cfc8-485f-acf8-c56389c13825-webhook-cert\") pod \"opendatahub-operator-controller-manager-797557847b-5kp5j\" (UID: \"8a0e0403-cfc8-485f-acf8-c56389c13825\") " pod="opendatahub/opendatahub-operator-controller-manager-797557847b-5kp5j" Sep 10 03:56:34.105454 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:34.105432 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xcw8\" (UniqueName: \"kubernetes.io/projected/8a0e0403-cfc8-485f-acf8-c56389c13825-kube-api-access-2xcw8\") pod \"opendatahub-operator-controller-manager-797557847b-5kp5j\" (UID: \"8a0e0403-cfc8-485f-acf8-c56389c13825\") " pod="opendatahub/opendatahub-operator-controller-manager-797557847b-5kp5j" Sep 10 03:56:34.227528 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:34.227437 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-797557847b-5kp5j" Sep 10 03:56:34.502620 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:34.502540 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-797557847b-5kp5j"] Sep 10 03:56:34.505789 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:56:34.505760 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a0e0403_cfc8_485f_acf8_c56389c13825.slice/crio-235b3ceefd1778d87bd9b3177d38852139d41e73585f7c0f848e85fcba2f3528 WatchSource:0}: Error finding container 235b3ceefd1778d87bd9b3177d38852139d41e73585f7c0f848e85fcba2f3528: Status 404 returned error can't find the container with id 235b3ceefd1778d87bd9b3177d38852139d41e73585f7c0f848e85fcba2f3528 Sep 10 03:56:34.618669 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:34.618638 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-797557847b-5kp5j" event={"ID":"8a0e0403-cfc8-485f-acf8-c56389c13825","Type":"ContainerStarted","Data":"235b3ceefd1778d87bd9b3177d38852139d41e73585f7c0f848e85fcba2f3528"} Sep 10 03:56:38.634449 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:38.634409 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-797557847b-5kp5j" event={"ID":"8a0e0403-cfc8-485f-acf8-c56389c13825","Type":"ContainerStarted","Data":"b049f7c7eae5d4d09d2153a392e5889bdc989b904ea83cb0f8ed2a84f2f49bde"} Sep 10 03:56:39.639424 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:39.639388 2581 generic.go:356] "Generic (PLEG): container finished" podID="8a0e0403-cfc8-485f-acf8-c56389c13825" containerID="b049f7c7eae5d4d09d2153a392e5889bdc989b904ea83cb0f8ed2a84f2f49bde" exitCode=0 Sep 10 03:56:39.639814 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:39.639474 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-797557847b-5kp5j" event={"ID":"8a0e0403-cfc8-485f-acf8-c56389c13825","Type":"ContainerDied","Data":"b049f7c7eae5d4d09d2153a392e5889bdc989b904ea83cb0f8ed2a84f2f49bde"} Sep 10 03:56:40.644784 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:40.644745 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-797557847b-5kp5j" event={"ID":"8a0e0403-cfc8-485f-acf8-c56389c13825","Type":"ContainerStarted","Data":"709dedc94e2c57ea5182d697289df4cc8f341ef330bb97a75c0f885b4e1b9485"} Sep 10 03:56:40.645207 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:40.644878 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-797557847b-5kp5j" Sep 10 03:56:40.667351 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:40.667310 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-797557847b-5kp5j" podStartSLOduration=3.115524229 podStartE2EDuration="7.667295159s" totalImagesPullingTime="3.484972735s" totalInitContainerRuntime="1.066798195s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:56:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:56:40.666307085 +0000 UTC m=+514.191686189" watchObservedRunningTime="2026-09-10 03:56:40.667295159 +0000 UTC m=+514.192674246" Sep 10 03:56:51.540075 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:51.540041 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-78c97bdccc-7zmx8"] Sep 10 03:56:51.545267 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:51.545248 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-78c97bdccc-7zmx8" Sep 10 03:56:51.547597 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:51.547575 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Sep 10 03:56:51.547717 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:51.547606 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Sep 10 03:56:51.547717 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:51.547632 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Sep 10 03:56:51.548391 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:51.548377 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-9rsl5\"" Sep 10 03:56:51.548442 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:51.548380 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Sep 10 03:56:51.548442 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:51.548417 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Sep 10 03:56:51.551129 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:51.551087 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-78c97bdccc-7zmx8"] Sep 10 03:56:51.629296 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:51.629265 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/15ab54e8-0ae3-49e2-8eaf-ecc8ca8bf39b-cert\") pod \"lws-controller-manager-78c97bdccc-7zmx8\" (UID: \"15ab54e8-0ae3-49e2-8eaf-ecc8ca8bf39b\") " pod="openshift-lws-operator/lws-controller-manager-78c97bdccc-7zmx8" Sep 10 03:56:51.629296 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:51.629299 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/15ab54e8-0ae3-49e2-8eaf-ecc8ca8bf39b-metrics-cert\") pod \"lws-controller-manager-78c97bdccc-7zmx8\" (UID: \"15ab54e8-0ae3-49e2-8eaf-ecc8ca8bf39b\") " pod="openshift-lws-operator/lws-controller-manager-78c97bdccc-7zmx8" Sep 10 03:56:51.629495 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:51.629366 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqd4h\" (UniqueName: \"kubernetes.io/projected/15ab54e8-0ae3-49e2-8eaf-ecc8ca8bf39b-kube-api-access-tqd4h\") pod \"lws-controller-manager-78c97bdccc-7zmx8\" (UID: \"15ab54e8-0ae3-49e2-8eaf-ecc8ca8bf39b\") " pod="openshift-lws-operator/lws-controller-manager-78c97bdccc-7zmx8" Sep 10 03:56:51.629495 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:51.629407 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/15ab54e8-0ae3-49e2-8eaf-ecc8ca8bf39b-manager-config\") pod \"lws-controller-manager-78c97bdccc-7zmx8\" (UID: \"15ab54e8-0ae3-49e2-8eaf-ecc8ca8bf39b\") " pod="openshift-lws-operator/lws-controller-manager-78c97bdccc-7zmx8" Sep 10 03:56:51.650888 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:51.650858 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-797557847b-5kp5j" Sep 10 03:56:51.730634 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:51.730604 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/15ab54e8-0ae3-49e2-8eaf-ecc8ca8bf39b-cert\") pod \"lws-controller-manager-78c97bdccc-7zmx8\" (UID: \"15ab54e8-0ae3-49e2-8eaf-ecc8ca8bf39b\") " pod="openshift-lws-operator/lws-controller-manager-78c97bdccc-7zmx8" Sep 10 03:56:51.730806 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:51.730644 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/15ab54e8-0ae3-49e2-8eaf-ecc8ca8bf39b-metrics-cert\") pod \"lws-controller-manager-78c97bdccc-7zmx8\" (UID: \"15ab54e8-0ae3-49e2-8eaf-ecc8ca8bf39b\") " pod="openshift-lws-operator/lws-controller-manager-78c97bdccc-7zmx8" Sep 10 03:56:51.730889 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:51.730861 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqd4h\" (UniqueName: \"kubernetes.io/projected/15ab54e8-0ae3-49e2-8eaf-ecc8ca8bf39b-kube-api-access-tqd4h\") pod \"lws-controller-manager-78c97bdccc-7zmx8\" (UID: \"15ab54e8-0ae3-49e2-8eaf-ecc8ca8bf39b\") " pod="openshift-lws-operator/lws-controller-manager-78c97bdccc-7zmx8" Sep 10 03:56:51.731144 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:51.731122 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/15ab54e8-0ae3-49e2-8eaf-ecc8ca8bf39b-manager-config\") pod \"lws-controller-manager-78c97bdccc-7zmx8\" (UID: \"15ab54e8-0ae3-49e2-8eaf-ecc8ca8bf39b\") " pod="openshift-lws-operator/lws-controller-manager-78c97bdccc-7zmx8" Sep 10 03:56:51.731820 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:51.731798 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/15ab54e8-0ae3-49e2-8eaf-ecc8ca8bf39b-manager-config\") pod \"lws-controller-manager-78c97bdccc-7zmx8\" (UID: \"15ab54e8-0ae3-49e2-8eaf-ecc8ca8bf39b\") " pod="openshift-lws-operator/lws-controller-manager-78c97bdccc-7zmx8" Sep 10 03:56:51.733525 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:51.733464 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/15ab54e8-0ae3-49e2-8eaf-ecc8ca8bf39b-metrics-cert\") pod \"lws-controller-manager-78c97bdccc-7zmx8\" (UID: \"15ab54e8-0ae3-49e2-8eaf-ecc8ca8bf39b\") " pod="openshift-lws-operator/lws-controller-manager-78c97bdccc-7zmx8" Sep 10 03:56:51.733630 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:51.733578 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/15ab54e8-0ae3-49e2-8eaf-ecc8ca8bf39b-cert\") pod \"lws-controller-manager-78c97bdccc-7zmx8\" (UID: \"15ab54e8-0ae3-49e2-8eaf-ecc8ca8bf39b\") " pod="openshift-lws-operator/lws-controller-manager-78c97bdccc-7zmx8" Sep 10 03:56:51.740063 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:51.740043 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqd4h\" (UniqueName: \"kubernetes.io/projected/15ab54e8-0ae3-49e2-8eaf-ecc8ca8bf39b-kube-api-access-tqd4h\") pod \"lws-controller-manager-78c97bdccc-7zmx8\" (UID: \"15ab54e8-0ae3-49e2-8eaf-ecc8ca8bf39b\") " pod="openshift-lws-operator/lws-controller-manager-78c97bdccc-7zmx8" Sep 10 03:56:51.855078 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:51.854993 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-78c97bdccc-7zmx8" Sep 10 03:56:52.124950 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:52.124924 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-78c97bdccc-7zmx8"] Sep 10 03:56:52.127177 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:56:52.127152 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15ab54e8_0ae3_49e2_8eaf_ecc8ca8bf39b.slice/crio-7cb7561a2bcd16f342000a52fc3315ae6b085dd88c19978e1fae788f2ca42c06 WatchSource:0}: Error finding container 7cb7561a2bcd16f342000a52fc3315ae6b085dd88c19978e1fae788f2ca42c06: Status 404 returned error can't find the container with id 7cb7561a2bcd16f342000a52fc3315ae6b085dd88c19978e1fae788f2ca42c06 Sep 10 03:56:52.686942 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:52.686906 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-78c97bdccc-7zmx8" event={"ID":"15ab54e8-0ae3-49e2-8eaf-ecc8ca8bf39b","Type":"ContainerStarted","Data":"7cb7561a2bcd16f342000a52fc3315ae6b085dd88c19978e1fae788f2ca42c06"} Sep 10 03:56:56.703919 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:56.703880 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-78c97bdccc-7zmx8" event={"ID":"15ab54e8-0ae3-49e2-8eaf-ecc8ca8bf39b","Type":"ContainerStarted","Data":"d1cd4e63ade532060f1240ee63eccea76112438d4863266ebab3ca1e2c707c61"} Sep 10 03:56:56.704323 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:56.704018 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-78c97bdccc-7zmx8" Sep 10 03:56:56.719040 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:56:56.718991 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-78c97bdccc-7zmx8" podStartSLOduration=2.100703317 podStartE2EDuration="5.718961287s" totalImagesPullingTime="3.61825797s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:56:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:56:56.718756744 +0000 UTC m=+530.244135849" watchObservedRunningTime="2026-09-10 03:56:56.718961287 +0000 UTC m=+530.244340390" Sep 10 03:57:00.230655 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:00.230574 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799chwtkx"] Sep 10 03:57:00.233610 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:00.233590 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799chwtkx" Sep 10 03:57:00.236230 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:00.236207 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 10 03:57:00.236372 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:00.236353 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 10 03:57:00.236519 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:00.236503 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-psc7r\"" Sep 10 03:57:00.236617 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:00.236570 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 10 03:57:00.244527 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:00.244505 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799chwtkx"] Sep 10 03:57:00.402575 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:00.402541 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db5782e3-ade9-48e9-8e2c-2aa4e36c4c15-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799chwtkx\" (UID: \"db5782e3-ade9-48e9-8e2c-2aa4e36c4c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799chwtkx" Sep 10 03:57:00.402575 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:00.402575 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db5782e3-ade9-48e9-8e2c-2aa4e36c4c15-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799chwtkx\" (UID: \"db5782e3-ade9-48e9-8e2c-2aa4e36c4c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799chwtkx" Sep 10 03:57:00.402832 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:00.402606 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db5782e3-ade9-48e9-8e2c-2aa4e36c4c15-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799chwtkx\" (UID: \"db5782e3-ade9-48e9-8e2c-2aa4e36c4c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799chwtkx" Sep 10 03:57:00.402832 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:00.402624 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db5782e3-ade9-48e9-8e2c-2aa4e36c4c15-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799chwtkx\" (UID: \"db5782e3-ade9-48e9-8e2c-2aa4e36c4c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799chwtkx" Sep 10 03:57:00.402832 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:00.402671 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db5782e3-ade9-48e9-8e2c-2aa4e36c4c15-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799chwtkx\" (UID: \"db5782e3-ade9-48e9-8e2c-2aa4e36c4c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799chwtkx" Sep 10 03:57:00.402832 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:00.402734 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db5782e3-ade9-48e9-8e2c-2aa4e36c4c15-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799chwtkx\" (UID: \"db5782e3-ade9-48e9-8e2c-2aa4e36c4c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799chwtkx" Sep 10 03:57:00.402832 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:00.402812 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxhls\" (UniqueName: \"kubernetes.io/projected/db5782e3-ade9-48e9-8e2c-2aa4e36c4c15-kube-api-access-cxhls\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799chwtkx\" (UID: \"db5782e3-ade9-48e9-8e2c-2aa4e36c4c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799chwtkx" Sep 10 03:57:00.403077 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:00.402851 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db5782e3-ade9-48e9-8e2c-2aa4e36c4c15-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799chwtkx\" (UID: \"db5782e3-ade9-48e9-8e2c-2aa4e36c4c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799chwtkx" Sep 10 03:57:00.403077 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:00.402899 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db5782e3-ade9-48e9-8e2c-2aa4e36c4c15-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799chwtkx\" (UID: \"db5782e3-ade9-48e9-8e2c-2aa4e36c4c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799chwtkx" Sep 10 03:57:00.503506 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:00.503400 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db5782e3-ade9-48e9-8e2c-2aa4e36c4c15-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799chwtkx\" (UID: \"db5782e3-ade9-48e9-8e2c-2aa4e36c4c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799chwtkx" Sep 10 03:57:00.503506 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:00.503457 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db5782e3-ade9-48e9-8e2c-2aa4e36c4c15-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799chwtkx\" (UID: \"db5782e3-ade9-48e9-8e2c-2aa4e36c4c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799chwtkx" Sep 10 03:57:00.503506 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:00.503482 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db5782e3-ade9-48e9-8e2c-2aa4e36c4c15-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799chwtkx\" (UID: \"db5782e3-ade9-48e9-8e2c-2aa4e36c4c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799chwtkx" Sep 10 03:57:00.503506 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:00.503505 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db5782e3-ade9-48e9-8e2c-2aa4e36c4c15-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799chwtkx\" (UID: \"db5782e3-ade9-48e9-8e2c-2aa4e36c4c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799chwtkx" Sep 10 03:57:00.503827 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:00.503538 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxhls\" (UniqueName: \"kubernetes.io/projected/db5782e3-ade9-48e9-8e2c-2aa4e36c4c15-kube-api-access-cxhls\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799chwtkx\" (UID: \"db5782e3-ade9-48e9-8e2c-2aa4e36c4c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799chwtkx" Sep 10 03:57:00.503827 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:00.503569 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db5782e3-ade9-48e9-8e2c-2aa4e36c4c15-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799chwtkx\" (UID: \"db5782e3-ade9-48e9-8e2c-2aa4e36c4c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799chwtkx" Sep 10 03:57:00.503827 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:00.503602 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db5782e3-ade9-48e9-8e2c-2aa4e36c4c15-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799chwtkx\" (UID: \"db5782e3-ade9-48e9-8e2c-2aa4e36c4c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799chwtkx" Sep 10 03:57:00.503827 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:00.503644 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db5782e3-ade9-48e9-8e2c-2aa4e36c4c15-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799chwtkx\" (UID: \"db5782e3-ade9-48e9-8e2c-2aa4e36c4c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799chwtkx" Sep 10 03:57:00.503827 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:00.503666 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db5782e3-ade9-48e9-8e2c-2aa4e36c4c15-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799chwtkx\" (UID: \"db5782e3-ade9-48e9-8e2c-2aa4e36c4c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799chwtkx" Sep 10 03:57:00.504106 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:00.503910 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db5782e3-ade9-48e9-8e2c-2aa4e36c4c15-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799chwtkx\" (UID: \"db5782e3-ade9-48e9-8e2c-2aa4e36c4c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799chwtkx" Sep 10 03:57:00.504106 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:00.504024 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db5782e3-ade9-48e9-8e2c-2aa4e36c4c15-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799chwtkx\" (UID: \"db5782e3-ade9-48e9-8e2c-2aa4e36c4c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799chwtkx" Sep 10 03:57:00.504382 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:00.504129 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db5782e3-ade9-48e9-8e2c-2aa4e36c4c15-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799chwtkx\" (UID: \"db5782e3-ade9-48e9-8e2c-2aa4e36c4c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799chwtkx" Sep 10 03:57:00.504382 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:00.504275 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db5782e3-ade9-48e9-8e2c-2aa4e36c4c15-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799chwtkx\" (UID: \"db5782e3-ade9-48e9-8e2c-2aa4e36c4c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799chwtkx" Sep 10 03:57:00.504484 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:00.504406 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db5782e3-ade9-48e9-8e2c-2aa4e36c4c15-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799chwtkx\" (UID: \"db5782e3-ade9-48e9-8e2c-2aa4e36c4c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799chwtkx" Sep 10 03:57:00.505955 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:00.505934 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db5782e3-ade9-48e9-8e2c-2aa4e36c4c15-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799chwtkx\" (UID: \"db5782e3-ade9-48e9-8e2c-2aa4e36c4c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799chwtkx" Sep 10 03:57:00.506226 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:00.506208 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db5782e3-ade9-48e9-8e2c-2aa4e36c4c15-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799chwtkx\" (UID: \"db5782e3-ade9-48e9-8e2c-2aa4e36c4c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799chwtkx" Sep 10 03:57:00.511336 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:00.511303 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db5782e3-ade9-48e9-8e2c-2aa4e36c4c15-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799chwtkx\" (UID: \"db5782e3-ade9-48e9-8e2c-2aa4e36c4c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799chwtkx" Sep 10 03:57:00.511513 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:00.511493 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxhls\" (UniqueName: \"kubernetes.io/projected/db5782e3-ade9-48e9-8e2c-2aa4e36c4c15-kube-api-access-cxhls\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799chwtkx\" (UID: \"db5782e3-ade9-48e9-8e2c-2aa4e36c4c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799chwtkx" Sep 10 03:57:00.544119 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:00.544086 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799chwtkx" Sep 10 03:57:00.822046 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:57:00.822013 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb5782e3_ade9_48e9_8e2c_2aa4e36c4c15.slice/crio-6792be133203acd667cbe8095e05c65d82828c30b67d2bb959488c5b920fa6c6 WatchSource:0}: Error finding container 6792be133203acd667cbe8095e05c65d82828c30b67d2bb959488c5b920fa6c6: Status 404 returned error can't find the container with id 6792be133203acd667cbe8095e05c65d82828c30b67d2bb959488c5b920fa6c6 Sep 10 03:57:00.822429 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:00.822406 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799chwtkx"] Sep 10 03:57:01.721537 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:01.721505 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799chwtkx" event={"ID":"db5782e3-ade9-48e9-8e2c-2aa4e36c4c15","Type":"ContainerStarted","Data":"6792be133203acd667cbe8095e05c65d82828c30b67d2bb959488c5b920fa6c6"} Sep 10 03:57:03.677604 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:03.677565 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 10 03:57:03.677845 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:03.677644 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 10 03:57:03.677845 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:03.677676 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 10 03:57:04.256521 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:04.256475 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-lzd2s"] Sep 10 03:57:04.259617 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:04.259602 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-lzd2s" Sep 10 03:57:04.262199 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:04.262176 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Sep 10 03:57:04.262341 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:04.262200 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-vm8xk\"" Sep 10 03:57:04.270260 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:04.270234 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-lzd2s"] Sep 10 03:57:04.441694 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:04.441651 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7j52x\" (UniqueName: \"kubernetes.io/projected/74e98872-0bca-4109-9865-2611f1493cde-kube-api-access-7j52x\") pod \"odh-model-controller-7998c67dc4-lzd2s\" (UID: \"74e98872-0bca-4109-9865-2611f1493cde\") " pod="opendatahub/odh-model-controller-7998c67dc4-lzd2s" Sep 10 03:57:04.441850 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:04.441752 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/74e98872-0bca-4109-9865-2611f1493cde-cert\") pod \"odh-model-controller-7998c67dc4-lzd2s\" (UID: \"74e98872-0bca-4109-9865-2611f1493cde\") " pod="opendatahub/odh-model-controller-7998c67dc4-lzd2s" Sep 10 03:57:04.542401 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:04.542348 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/74e98872-0bca-4109-9865-2611f1493cde-cert\") pod \"odh-model-controller-7998c67dc4-lzd2s\" (UID: \"74e98872-0bca-4109-9865-2611f1493cde\") " pod="opendatahub/odh-model-controller-7998c67dc4-lzd2s" Sep 10 03:57:04.542571 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:04.542437 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7j52x\" (UniqueName: \"kubernetes.io/projected/74e98872-0bca-4109-9865-2611f1493cde-kube-api-access-7j52x\") pod \"odh-model-controller-7998c67dc4-lzd2s\" (UID: \"74e98872-0bca-4109-9865-2611f1493cde\") " pod="opendatahub/odh-model-controller-7998c67dc4-lzd2s" Sep 10 03:57:04.542571 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:57:04.542509 2581 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Sep 10 03:57:04.542571 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:57:04.542564 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74e98872-0bca-4109-9865-2611f1493cde-cert podName:74e98872-0bca-4109-9865-2611f1493cde nodeName:}" failed. No retries permitted until 2026-09-10 03:57:05.042549151 +0000 UTC m=+538.567928235 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/74e98872-0bca-4109-9865-2611f1493cde-cert") pod "odh-model-controller-7998c67dc4-lzd2s" (UID: "74e98872-0bca-4109-9865-2611f1493cde") : secret "odh-model-controller-webhook-cert" not found Sep 10 03:57:04.550636 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:04.550613 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7j52x\" (UniqueName: \"kubernetes.io/projected/74e98872-0bca-4109-9865-2611f1493cde-kube-api-access-7j52x\") pod \"odh-model-controller-7998c67dc4-lzd2s\" (UID: \"74e98872-0bca-4109-9865-2611f1493cde\") " pod="opendatahub/odh-model-controller-7998c67dc4-lzd2s" Sep 10 03:57:04.734417 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:04.734374 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799chwtkx" event={"ID":"db5782e3-ade9-48e9-8e2c-2aa4e36c4c15","Type":"ContainerStarted","Data":"7cd0cc15405966adea160a26a874a34bc6322f7d0ab48602648b9553f0c8009f"} Sep 10 03:57:04.756667 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:04.756610 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799chwtkx" podStartSLOduration=1.903427424 podStartE2EDuration="4.75659032s" totalImagesPullingTime="2.853162896s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:57:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:57:04.755359532 +0000 UTC m=+538.280738647" watchObservedRunningTime="2026-09-10 03:57:04.75659032 +0000 UTC m=+538.281969426" Sep 10 03:57:04.759749 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:04.759726 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-psjjd"] Sep 10 03:57:04.762541 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:04.762524 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-psjjd" Sep 10 03:57:04.764729 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:04.764709 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Sep 10 03:57:04.764829 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:04.764811 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-q25sg\"" Sep 10 03:57:04.771263 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:04.771241 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-psjjd"] Sep 10 03:57:04.945387 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:04.945313 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2c55df8-569f-4e2f-a317-ad282f00bddc-tls-certs\") pod \"model-serving-api-54d89bd5fc-psjjd\" (UID: \"e2c55df8-569f-4e2f-a317-ad282f00bddc\") " pod="opendatahub/model-serving-api-54d89bd5fc-psjjd" Sep 10 03:57:04.945387 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:04.945348 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrhwk\" (UniqueName: \"kubernetes.io/projected/e2c55df8-569f-4e2f-a317-ad282f00bddc-kube-api-access-rrhwk\") pod \"model-serving-api-54d89bd5fc-psjjd\" (UID: \"e2c55df8-569f-4e2f-a317-ad282f00bddc\") " pod="opendatahub/model-serving-api-54d89bd5fc-psjjd" Sep 10 03:57:05.046892 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:05.046857 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrhwk\" (UniqueName: \"kubernetes.io/projected/e2c55df8-569f-4e2f-a317-ad282f00bddc-kube-api-access-rrhwk\") pod \"model-serving-api-54d89bd5fc-psjjd\" (UID: \"e2c55df8-569f-4e2f-a317-ad282f00bddc\") " pod="opendatahub/model-serving-api-54d89bd5fc-psjjd" Sep 10 03:57:05.047129 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:05.047104 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/74e98872-0bca-4109-9865-2611f1493cde-cert\") pod \"odh-model-controller-7998c67dc4-lzd2s\" (UID: \"74e98872-0bca-4109-9865-2611f1493cde\") " pod="opendatahub/odh-model-controller-7998c67dc4-lzd2s" Sep 10 03:57:05.047205 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:05.047153 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2c55df8-569f-4e2f-a317-ad282f00bddc-tls-certs\") pod \"model-serving-api-54d89bd5fc-psjjd\" (UID: \"e2c55df8-569f-4e2f-a317-ad282f00bddc\") " pod="opendatahub/model-serving-api-54d89bd5fc-psjjd" Sep 10 03:57:05.049534 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:05.049513 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/74e98872-0bca-4109-9865-2611f1493cde-cert\") pod \"odh-model-controller-7998c67dc4-lzd2s\" (UID: \"74e98872-0bca-4109-9865-2611f1493cde\") " pod="opendatahub/odh-model-controller-7998c67dc4-lzd2s" Sep 10 03:57:05.049649 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:05.049595 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2c55df8-569f-4e2f-a317-ad282f00bddc-tls-certs\") pod \"model-serving-api-54d89bd5fc-psjjd\" (UID: \"e2c55df8-569f-4e2f-a317-ad282f00bddc\") " pod="opendatahub/model-serving-api-54d89bd5fc-psjjd" Sep 10 03:57:05.054370 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:05.054341 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrhwk\" (UniqueName: \"kubernetes.io/projected/e2c55df8-569f-4e2f-a317-ad282f00bddc-kube-api-access-rrhwk\") pod \"model-serving-api-54d89bd5fc-psjjd\" (UID: \"e2c55df8-569f-4e2f-a317-ad282f00bddc\") " pod="opendatahub/model-serving-api-54d89bd5fc-psjjd" Sep 10 03:57:05.073326 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:05.073302 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-psjjd" Sep 10 03:57:05.169587 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:05.169556 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-lzd2s" Sep 10 03:57:05.342092 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:05.342064 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-psjjd"] Sep 10 03:57:05.344957 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:57:05.344928 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2c55df8_569f_4e2f_a317_ad282f00bddc.slice/crio-0b31559573f7ff5cd2a5e669ce2eec98561a76a4477654629fdb2bcf84c8eebd WatchSource:0}: Error finding container 0b31559573f7ff5cd2a5e669ce2eec98561a76a4477654629fdb2bcf84c8eebd: Status 404 returned error can't find the container with id 0b31559573f7ff5cd2a5e669ce2eec98561a76a4477654629fdb2bcf84c8eebd Sep 10 03:57:05.437953 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:05.437930 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-lzd2s"] Sep 10 03:57:05.440326 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:57:05.440299 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74e98872_0bca_4109_9865_2611f1493cde.slice/crio-dc6974362d8ba4e96be5be9c0a5bd6b568ee10d7059f8d4c4e4c0850b07bdcfc WatchSource:0}: Error finding container dc6974362d8ba4e96be5be9c0a5bd6b568ee10d7059f8d4c4e4c0850b07bdcfc: Status 404 returned error can't find the container with id dc6974362d8ba4e96be5be9c0a5bd6b568ee10d7059f8d4c4e4c0850b07bdcfc Sep 10 03:57:05.544470 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:05.544436 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799chwtkx" Sep 10 03:57:05.550309 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:05.550286 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799chwtkx" Sep 10 03:57:05.739616 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:05.739581 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-psjjd" event={"ID":"e2c55df8-569f-4e2f-a317-ad282f00bddc","Type":"ContainerStarted","Data":"0b31559573f7ff5cd2a5e669ce2eec98561a76a4477654629fdb2bcf84c8eebd"} Sep 10 03:57:05.740736 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:05.740708 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-lzd2s" event={"ID":"74e98872-0bca-4109-9865-2611f1493cde","Type":"ContainerStarted","Data":"dc6974362d8ba4e96be5be9c0a5bd6b568ee10d7059f8d4c4e4c0850b07bdcfc"} Sep 10 03:57:05.740932 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:05.740912 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799chwtkx" Sep 10 03:57:05.741848 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:05.741829 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799chwtkx" Sep 10 03:57:07.710168 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:07.710132 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-78c97bdccc-7zmx8" Sep 10 03:57:09.764753 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:09.764651 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-psjjd" event={"ID":"e2c55df8-569f-4e2f-a317-ad282f00bddc","Type":"ContainerStarted","Data":"7084c40890cb60468d23510fed15b9a9190164af8dd2d59e3c0914640e5e8d3b"} Sep 10 03:57:09.764753 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:09.764722 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-psjjd" Sep 10 03:57:09.766364 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:09.766342 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-lzd2s" event={"ID":"74e98872-0bca-4109-9865-2611f1493cde","Type":"ContainerStarted","Data":"e037f1f9a5489e6e7fada2e26e7c3bbbdd0e0b7c77219068876544e9466240fb"} Sep 10 03:57:09.766495 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:09.766480 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-lzd2s" Sep 10 03:57:09.794716 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:09.794646 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-psjjd" podStartSLOduration=1.965653816 podStartE2EDuration="5.794633132s" totalImagesPullingTime="3.828979316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:57:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:57:09.793596079 +0000 UTC m=+543.318975181" watchObservedRunningTime="2026-09-10 03:57:09.794633132 +0000 UTC m=+543.320012237" Sep 10 03:57:09.812604 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:09.812566 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-lzd2s" podStartSLOduration=1.805576074 podStartE2EDuration="5.812553225s" totalImagesPullingTime="4.006977151s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:57:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:57:09.81135802 +0000 UTC m=+543.336737128" watchObservedRunningTime="2026-09-10 03:57:09.812553225 +0000 UTC m=+543.337932329" Sep 10 03:57:11.053323 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:11.053289 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-8tmrn"] Sep 10 03:57:11.055414 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:11.055397 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8tmrn" Sep 10 03:57:11.057782 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:11.057753 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Sep 10 03:57:11.057913 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:11.057808 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Sep 10 03:57:11.058157 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:11.058138 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-vds9h\"" Sep 10 03:57:11.058479 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:11.058465 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Sep 10 03:57:11.069209 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:11.069190 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-8tmrn"] Sep 10 03:57:11.099821 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:11.099797 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8knqt\" (UniqueName: \"kubernetes.io/projected/c13ee959-189a-4d0a-adec-2f14afab064a-kube-api-access-8knqt\") pod \"istiod-openshift-gateway-5677ccb49b-8tmrn\" (UID: \"c13ee959-189a-4d0a-adec-2f14afab064a\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8tmrn" Sep 10 03:57:11.099940 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:11.099834 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c13ee959-189a-4d0a-adec-2f14afab064a-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-8tmrn\" (UID: \"c13ee959-189a-4d0a-adec-2f14afab064a\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8tmrn" Sep 10 03:57:11.099940 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:11.099864 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c13ee959-189a-4d0a-adec-2f14afab064a-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-8tmrn\" (UID: \"c13ee959-189a-4d0a-adec-2f14afab064a\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8tmrn" Sep 10 03:57:11.099940 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:11.099879 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c13ee959-189a-4d0a-adec-2f14afab064a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-8tmrn\" (UID: \"c13ee959-189a-4d0a-adec-2f14afab064a\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8tmrn" Sep 10 03:57:11.099940 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:11.099894 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c13ee959-189a-4d0a-adec-2f14afab064a-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-8tmrn\" (UID: \"c13ee959-189a-4d0a-adec-2f14afab064a\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8tmrn" Sep 10 03:57:11.099940 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:11.099908 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c13ee959-189a-4d0a-adec-2f14afab064a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-8tmrn\" (UID: \"c13ee959-189a-4d0a-adec-2f14afab064a\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8tmrn" Sep 10 03:57:11.099940 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:11.099929 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c13ee959-189a-4d0a-adec-2f14afab064a-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-8tmrn\" (UID: \"c13ee959-189a-4d0a-adec-2f14afab064a\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8tmrn" Sep 10 03:57:11.200408 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:11.200374 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8knqt\" (UniqueName: \"kubernetes.io/projected/c13ee959-189a-4d0a-adec-2f14afab064a-kube-api-access-8knqt\") pod \"istiod-openshift-gateway-5677ccb49b-8tmrn\" (UID: \"c13ee959-189a-4d0a-adec-2f14afab064a\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8tmrn" Sep 10 03:57:11.200698 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:11.200669 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c13ee959-189a-4d0a-adec-2f14afab064a-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-8tmrn\" (UID: \"c13ee959-189a-4d0a-adec-2f14afab064a\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8tmrn" Sep 10 03:57:11.200820 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:11.200740 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c13ee959-189a-4d0a-adec-2f14afab064a-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-8tmrn\" (UID: \"c13ee959-189a-4d0a-adec-2f14afab064a\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8tmrn" Sep 10 03:57:11.200820 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:11.200762 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c13ee959-189a-4d0a-adec-2f14afab064a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-8tmrn\" (UID: \"c13ee959-189a-4d0a-adec-2f14afab064a\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8tmrn" Sep 10 03:57:11.200820 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:11.200780 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c13ee959-189a-4d0a-adec-2f14afab064a-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-8tmrn\" (UID: \"c13ee959-189a-4d0a-adec-2f14afab064a\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8tmrn" Sep 10 03:57:11.200820 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:11.200802 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c13ee959-189a-4d0a-adec-2f14afab064a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-8tmrn\" (UID: \"c13ee959-189a-4d0a-adec-2f14afab064a\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8tmrn" Sep 10 03:57:11.201066 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:11.200835 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c13ee959-189a-4d0a-adec-2f14afab064a-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-8tmrn\" (UID: \"c13ee959-189a-4d0a-adec-2f14afab064a\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8tmrn" Sep 10 03:57:11.201602 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:11.201581 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c13ee959-189a-4d0a-adec-2f14afab064a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-8tmrn\" (UID: \"c13ee959-189a-4d0a-adec-2f14afab064a\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8tmrn" Sep 10 03:57:11.203162 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:11.203134 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c13ee959-189a-4d0a-adec-2f14afab064a-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-8tmrn\" (UID: \"c13ee959-189a-4d0a-adec-2f14afab064a\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8tmrn" Sep 10 03:57:11.203317 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:11.203298 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c13ee959-189a-4d0a-adec-2f14afab064a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-8tmrn\" (UID: \"c13ee959-189a-4d0a-adec-2f14afab064a\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8tmrn" Sep 10 03:57:11.204112 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:11.204096 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c13ee959-189a-4d0a-adec-2f14afab064a-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-8tmrn\" (UID: \"c13ee959-189a-4d0a-adec-2f14afab064a\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8tmrn" Sep 10 03:57:11.204215 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:11.204198 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c13ee959-189a-4d0a-adec-2f14afab064a-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-8tmrn\" (UID: \"c13ee959-189a-4d0a-adec-2f14afab064a\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8tmrn" Sep 10 03:57:11.208258 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:11.208236 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c13ee959-189a-4d0a-adec-2f14afab064a-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-8tmrn\" (UID: \"c13ee959-189a-4d0a-adec-2f14afab064a\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8tmrn" Sep 10 03:57:11.208469 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:11.208451 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8knqt\" (UniqueName: \"kubernetes.io/projected/c13ee959-189a-4d0a-adec-2f14afab064a-kube-api-access-8knqt\") pod \"istiod-openshift-gateway-5677ccb49b-8tmrn\" (UID: \"c13ee959-189a-4d0a-adec-2f14afab064a\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8tmrn" Sep 10 03:57:11.364694 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:11.364612 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8tmrn" Sep 10 03:57:11.639212 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:11.639139 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-8tmrn"] Sep 10 03:57:11.642478 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:57:11.642450 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc13ee959_189a_4d0a_adec_2f14afab064a.slice/crio-accd0fb11b4453091f8875c9f0661946cf9d5433c86fe68155ece4a1605884fc WatchSource:0}: Error finding container accd0fb11b4453091f8875c9f0661946cf9d5433c86fe68155ece4a1605884fc: Status 404 returned error can't find the container with id accd0fb11b4453091f8875c9f0661946cf9d5433c86fe68155ece4a1605884fc Sep 10 03:57:11.776663 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:11.776622 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8tmrn" event={"ID":"c13ee959-189a-4d0a-adec-2f14afab064a","Type":"ContainerStarted","Data":"accd0fb11b4453091f8875c9f0661946cf9d5433c86fe68155ece4a1605884fc"} Sep 10 03:57:16.184542 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:16.184509 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 10 03:57:16.184787 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:16.184575 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 10 03:57:16.800443 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:16.800401 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8tmrn" event={"ID":"c13ee959-189a-4d0a-adec-2f14afab064a","Type":"ContainerStarted","Data":"852e44028c86b9e6fe43a538c293f83b6db2d3888afe2514582e0cba4d1addb3"} Sep 10 03:57:16.800633 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:16.800537 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8tmrn" Sep 10 03:57:16.823105 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:16.823036 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8tmrn" podStartSLOduration=1.283448639 podStartE2EDuration="5.823016567s" totalImagesPullingTime="4.539567928s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:57:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:57:16.821076707 +0000 UTC m=+550.346455814" watchObservedRunningTime="2026-09-10 03:57:16.823016567 +0000 UTC m=+550.348395671" Sep 10 03:57:17.806208 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:17.806178 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8tmrn" Sep 10 03:57:20.774560 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:20.774531 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-lzd2s" Sep 10 03:57:20.776810 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:57:20.776791 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-psjjd" Sep 10 03:58:19.396710 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:19.396675 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7f44969586-qn22r"] Sep 10 03:58:19.399834 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:19.399814 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f44969586-qn22r" Sep 10 03:58:19.409360 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:19.409340 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f44969586-qn22r"] Sep 10 03:58:19.548254 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:19.548221 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed4eb216-8475-492d-905f-acbf98058a5f-service-ca\") pod \"console-7f44969586-qn22r\" (UID: \"ed4eb216-8475-492d-905f-acbf98058a5f\") " pod="openshift-console/console-7f44969586-qn22r" Sep 10 03:58:19.548254 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:19.548256 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed4eb216-8475-492d-905f-acbf98058a5f-console-config\") pod \"console-7f44969586-qn22r\" (UID: \"ed4eb216-8475-492d-905f-acbf98058a5f\") " pod="openshift-console/console-7f44969586-qn22r" Sep 10 03:58:19.548493 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:19.548277 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed4eb216-8475-492d-905f-acbf98058a5f-console-oauth-config\") pod \"console-7f44969586-qn22r\" (UID: \"ed4eb216-8475-492d-905f-acbf98058a5f\") " pod="openshift-console/console-7f44969586-qn22r" Sep 10 03:58:19.548493 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:19.548330 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed4eb216-8475-492d-905f-acbf98058a5f-trusted-ca-bundle\") pod \"console-7f44969586-qn22r\" (UID: \"ed4eb216-8475-492d-905f-acbf98058a5f\") " pod="openshift-console/console-7f44969586-qn22r" Sep 10 03:58:19.548493 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:19.548430 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed4eb216-8475-492d-905f-acbf98058a5f-console-serving-cert\") pod \"console-7f44969586-qn22r\" (UID: \"ed4eb216-8475-492d-905f-acbf98058a5f\") " pod="openshift-console/console-7f44969586-qn22r" Sep 10 03:58:19.548493 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:19.548466 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z86f\" (UniqueName: \"kubernetes.io/projected/ed4eb216-8475-492d-905f-acbf98058a5f-kube-api-access-9z86f\") pod \"console-7f44969586-qn22r\" (UID: \"ed4eb216-8475-492d-905f-acbf98058a5f\") " pod="openshift-console/console-7f44969586-qn22r" Sep 10 03:58:19.548659 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:19.548522 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed4eb216-8475-492d-905f-acbf98058a5f-oauth-serving-cert\") pod \"console-7f44969586-qn22r\" (UID: \"ed4eb216-8475-492d-905f-acbf98058a5f\") " pod="openshift-console/console-7f44969586-qn22r" Sep 10 03:58:19.649090 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:19.648999 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed4eb216-8475-492d-905f-acbf98058a5f-console-serving-cert\") pod \"console-7f44969586-qn22r\" (UID: \"ed4eb216-8475-492d-905f-acbf98058a5f\") " pod="openshift-console/console-7f44969586-qn22r" Sep 10 03:58:19.649090 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:19.649040 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z86f\" (UniqueName: \"kubernetes.io/projected/ed4eb216-8475-492d-905f-acbf98058a5f-kube-api-access-9z86f\") pod \"console-7f44969586-qn22r\" (UID: \"ed4eb216-8475-492d-905f-acbf98058a5f\") " pod="openshift-console/console-7f44969586-qn22r" Sep 10 03:58:19.649090 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:19.649060 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed4eb216-8475-492d-905f-acbf98058a5f-oauth-serving-cert\") pod \"console-7f44969586-qn22r\" (UID: \"ed4eb216-8475-492d-905f-acbf98058a5f\") " pod="openshift-console/console-7f44969586-qn22r" Sep 10 03:58:19.649090 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:19.649083 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed4eb216-8475-492d-905f-acbf98058a5f-service-ca\") pod \"console-7f44969586-qn22r\" (UID: \"ed4eb216-8475-492d-905f-acbf98058a5f\") " pod="openshift-console/console-7f44969586-qn22r" Sep 10 03:58:19.649399 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:19.649103 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed4eb216-8475-492d-905f-acbf98058a5f-console-config\") pod \"console-7f44969586-qn22r\" (UID: \"ed4eb216-8475-492d-905f-acbf98058a5f\") " pod="openshift-console/console-7f44969586-qn22r" Sep 10 03:58:19.649399 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:19.649121 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed4eb216-8475-492d-905f-acbf98058a5f-console-oauth-config\") pod \"console-7f44969586-qn22r\" (UID: \"ed4eb216-8475-492d-905f-acbf98058a5f\") " pod="openshift-console/console-7f44969586-qn22r" Sep 10 03:58:19.649399 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:19.649158 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed4eb216-8475-492d-905f-acbf98058a5f-trusted-ca-bundle\") pod \"console-7f44969586-qn22r\" (UID: \"ed4eb216-8475-492d-905f-acbf98058a5f\") " pod="openshift-console/console-7f44969586-qn22r" Sep 10 03:58:19.649924 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:19.649897 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed4eb216-8475-492d-905f-acbf98058a5f-console-config\") pod \"console-7f44969586-qn22r\" (UID: \"ed4eb216-8475-492d-905f-acbf98058a5f\") " pod="openshift-console/console-7f44969586-qn22r" Sep 10 03:58:19.650062 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:19.650005 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed4eb216-8475-492d-905f-acbf98058a5f-service-ca\") pod \"console-7f44969586-qn22r\" (UID: \"ed4eb216-8475-492d-905f-acbf98058a5f\") " pod="openshift-console/console-7f44969586-qn22r" Sep 10 03:58:19.650062 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:19.649897 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed4eb216-8475-492d-905f-acbf98058a5f-oauth-serving-cert\") pod \"console-7f44969586-qn22r\" (UID: \"ed4eb216-8475-492d-905f-acbf98058a5f\") " pod="openshift-console/console-7f44969586-qn22r" Sep 10 03:58:19.650062 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:19.650042 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed4eb216-8475-492d-905f-acbf98058a5f-trusted-ca-bundle\") pod \"console-7f44969586-qn22r\" (UID: \"ed4eb216-8475-492d-905f-acbf98058a5f\") " pod="openshift-console/console-7f44969586-qn22r" Sep 10 03:58:19.651744 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:19.651721 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed4eb216-8475-492d-905f-acbf98058a5f-console-oauth-config\") pod \"console-7f44969586-qn22r\" (UID: \"ed4eb216-8475-492d-905f-acbf98058a5f\") " pod="openshift-console/console-7f44969586-qn22r" Sep 10 03:58:19.651852 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:19.651835 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed4eb216-8475-492d-905f-acbf98058a5f-console-serving-cert\") pod \"console-7f44969586-qn22r\" (UID: \"ed4eb216-8475-492d-905f-acbf98058a5f\") " pod="openshift-console/console-7f44969586-qn22r" Sep 10 03:58:19.657054 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:19.657027 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z86f\" (UniqueName: \"kubernetes.io/projected/ed4eb216-8475-492d-905f-acbf98058a5f-kube-api-access-9z86f\") pod \"console-7f44969586-qn22r\" (UID: \"ed4eb216-8475-492d-905f-acbf98058a5f\") " pod="openshift-console/console-7f44969586-qn22r" Sep 10 03:58:19.709641 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:19.709617 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f44969586-qn22r" Sep 10 03:58:19.979492 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:19.979466 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f44969586-qn22r"] Sep 10 03:58:19.982356 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:58:19.982333 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded4eb216_8475_492d_905f_acbf98058a5f.slice/crio-4614354f49dc199e41ad45711ebc65d1070f6ae233b16b0e0239506a2377793c WatchSource:0}: Error finding container 4614354f49dc199e41ad45711ebc65d1070f6ae233b16b0e0239506a2377793c: Status 404 returned error can't find the container with id 4614354f49dc199e41ad45711ebc65d1070f6ae233b16b0e0239506a2377793c Sep 10 03:58:20.053369 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:20.053340 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f44969586-qn22r" event={"ID":"ed4eb216-8475-492d-905f-acbf98058a5f","Type":"ContainerStarted","Data":"e795d6ce8f398b8376ae9289f7f964877bc32b1f0d005258f412f4241f89a63f"} Sep 10 03:58:20.053500 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:20.053392 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f44969586-qn22r" event={"ID":"ed4eb216-8475-492d-905f-acbf98058a5f","Type":"ContainerStarted","Data":"4614354f49dc199e41ad45711ebc65d1070f6ae233b16b0e0239506a2377793c"} Sep 10 03:58:20.072474 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:20.072420 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7f44969586-qn22r" podStartSLOduration=1.072403268 podStartE2EDuration="1.072403268s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:58:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:58:20.070552147 +0000 UTC m=+613.595931246" watchObservedRunningTime="2026-09-10 03:58:20.072403268 +0000 UTC m=+613.597782372" Sep 10 03:58:29.710552 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:29.710501 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7f44969586-qn22r" Sep 10 03:58:29.710955 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:29.710574 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7f44969586-qn22r" Sep 10 03:58:29.715470 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:29.715448 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7f44969586-qn22r" Sep 10 03:58:30.096353 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:30.096320 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7f44969586-qn22r" Sep 10 03:58:30.146742 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:30.146706 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cdfb499fc-592gv"] Sep 10 03:58:55.167708 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:55.167645 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-cdfb499fc-592gv" podUID="f28d5cf1-f8b6-4eb2-9b96-a74b97085b39" containerName="console" containerID="cri-o://28318e8db41e1065828de89c28b2e5dc4542ee2c4670f276aaa5477e57d60e6f" gracePeriod=15 Sep 10 03:58:55.401190 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:55.401167 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cdfb499fc-592gv" Sep 10 03:58:55.535171 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:55.535141 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f28d5cf1-f8b6-4eb2-9b96-a74b97085b39-console-serving-cert\") pod \"f28d5cf1-f8b6-4eb2-9b96-a74b97085b39\" (UID: \"f28d5cf1-f8b6-4eb2-9b96-a74b97085b39\") " Sep 10 03:58:55.535353 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:55.535190 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f28d5cf1-f8b6-4eb2-9b96-a74b97085b39-trusted-ca-bundle\") pod \"f28d5cf1-f8b6-4eb2-9b96-a74b97085b39\" (UID: \"f28d5cf1-f8b6-4eb2-9b96-a74b97085b39\") " Sep 10 03:58:55.535353 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:55.535214 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f28d5cf1-f8b6-4eb2-9b96-a74b97085b39-console-oauth-config\") pod \"f28d5cf1-f8b6-4eb2-9b96-a74b97085b39\" (UID: \"f28d5cf1-f8b6-4eb2-9b96-a74b97085b39\") " Sep 10 03:58:55.535353 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:55.535238 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f28d5cf1-f8b6-4eb2-9b96-a74b97085b39-console-config\") pod \"f28d5cf1-f8b6-4eb2-9b96-a74b97085b39\" (UID: \"f28d5cf1-f8b6-4eb2-9b96-a74b97085b39\") " Sep 10 03:58:55.535510 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:55.535368 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9pbrd\" (UniqueName: \"kubernetes.io/projected/f28d5cf1-f8b6-4eb2-9b96-a74b97085b39-kube-api-access-9pbrd\") pod \"f28d5cf1-f8b6-4eb2-9b96-a74b97085b39\" (UID: \"f28d5cf1-f8b6-4eb2-9b96-a74b97085b39\") " Sep 10 03:58:55.535510 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:55.535436 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f28d5cf1-f8b6-4eb2-9b96-a74b97085b39-oauth-serving-cert\") pod \"f28d5cf1-f8b6-4eb2-9b96-a74b97085b39\" (UID: \"f28d5cf1-f8b6-4eb2-9b96-a74b97085b39\") " Sep 10 03:58:55.535605 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:55.535522 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f28d5cf1-f8b6-4eb2-9b96-a74b97085b39-service-ca\") pod \"f28d5cf1-f8b6-4eb2-9b96-a74b97085b39\" (UID: \"f28d5cf1-f8b6-4eb2-9b96-a74b97085b39\") " Sep 10 03:58:55.535651 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:55.535623 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f28d5cf1-f8b6-4eb2-9b96-a74b97085b39-console-config" (OuterVolumeSpecName: "console-config") pod "f28d5cf1-f8b6-4eb2-9b96-a74b97085b39" (UID: "f28d5cf1-f8b6-4eb2-9b96-a74b97085b39"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 03:58:55.535651 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:55.535642 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f28d5cf1-f8b6-4eb2-9b96-a74b97085b39-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "f28d5cf1-f8b6-4eb2-9b96-a74b97085b39" (UID: "f28d5cf1-f8b6-4eb2-9b96-a74b97085b39"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 03:58:55.535876 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:55.535860 2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f28d5cf1-f8b6-4eb2-9b96-a74b97085b39-trusted-ca-bundle\") on node \"ip-10-0-142-77.ec2.internal\" DevicePath \"\"" Sep 10 03:58:55.535876 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:55.535878 2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f28d5cf1-f8b6-4eb2-9b96-a74b97085b39-console-config\") on node \"ip-10-0-142-77.ec2.internal\" DevicePath \"\"" Sep 10 03:58:55.536027 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:55.535833 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f28d5cf1-f8b6-4eb2-9b96-a74b97085b39-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f28d5cf1-f8b6-4eb2-9b96-a74b97085b39" (UID: "f28d5cf1-f8b6-4eb2-9b96-a74b97085b39"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 03:58:55.536027 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:55.535889 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f28d5cf1-f8b6-4eb2-9b96-a74b97085b39-service-ca" (OuterVolumeSpecName: "service-ca") pod "f28d5cf1-f8b6-4eb2-9b96-a74b97085b39" (UID: "f28d5cf1-f8b6-4eb2-9b96-a74b97085b39"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 03:58:55.537434 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:55.537412 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f28d5cf1-f8b6-4eb2-9b96-a74b97085b39-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f28d5cf1-f8b6-4eb2-9b96-a74b97085b39" (UID: "f28d5cf1-f8b6-4eb2-9b96-a74b97085b39"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 03:58:55.538120 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:55.538090 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f28d5cf1-f8b6-4eb2-9b96-a74b97085b39-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f28d5cf1-f8b6-4eb2-9b96-a74b97085b39" (UID: "f28d5cf1-f8b6-4eb2-9b96-a74b97085b39"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 03:58:55.538120 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:55.538104 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f28d5cf1-f8b6-4eb2-9b96-a74b97085b39-kube-api-access-9pbrd" (OuterVolumeSpecName: "kube-api-access-9pbrd") pod "f28d5cf1-f8b6-4eb2-9b96-a74b97085b39" (UID: "f28d5cf1-f8b6-4eb2-9b96-a74b97085b39"). InnerVolumeSpecName "kube-api-access-9pbrd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 03:58:55.636899 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:55.636867 2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f28d5cf1-f8b6-4eb2-9b96-a74b97085b39-console-serving-cert\") on node \"ip-10-0-142-77.ec2.internal\" DevicePath \"\"" Sep 10 03:58:55.636899 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:55.636891 2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f28d5cf1-f8b6-4eb2-9b96-a74b97085b39-console-oauth-config\") on node \"ip-10-0-142-77.ec2.internal\" DevicePath \"\"" Sep 10 03:58:55.636899 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:55.636901 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9pbrd\" (UniqueName: \"kubernetes.io/projected/f28d5cf1-f8b6-4eb2-9b96-a74b97085b39-kube-api-access-9pbrd\") on node \"ip-10-0-142-77.ec2.internal\" DevicePath \"\"" Sep 10 03:58:55.637145 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:55.636909 2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f28d5cf1-f8b6-4eb2-9b96-a74b97085b39-oauth-serving-cert\") on node \"ip-10-0-142-77.ec2.internal\" DevicePath \"\"" Sep 10 03:58:55.637145 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:55.636919 2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f28d5cf1-f8b6-4eb2-9b96-a74b97085b39-service-ca\") on node \"ip-10-0-142-77.ec2.internal\" DevicePath \"\"" Sep 10 03:58:56.201681 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:56.201642 2581 generic.go:356] "Generic (PLEG): container finished" podID="f28d5cf1-f8b6-4eb2-9b96-a74b97085b39" containerID="28318e8db41e1065828de89c28b2e5dc4542ee2c4670f276aaa5477e57d60e6f" exitCode=0 Sep 10 03:58:56.201681 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:56.201688 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cdfb499fc-592gv" Sep 10 03:58:56.201681 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:56.201694 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cdfb499fc-592gv" event={"ID":"f28d5cf1-f8b6-4eb2-9b96-a74b97085b39","Type":"ContainerDied","Data":"28318e8db41e1065828de89c28b2e5dc4542ee2c4670f276aaa5477e57d60e6f"} Sep 10 03:58:56.202220 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:56.201772 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cdfb499fc-592gv" event={"ID":"f28d5cf1-f8b6-4eb2-9b96-a74b97085b39","Type":"ContainerDied","Data":"30e370af756eaa5189eb1d4aa6976882c2a3c688465b28f18ff6edde702aa86e"} Sep 10 03:58:56.202220 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:56.201799 2581 scope.go:117] "RemoveContainer" containerID="28318e8db41e1065828de89c28b2e5dc4542ee2c4670f276aaa5477e57d60e6f" Sep 10 03:58:56.210740 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:56.210721 2581 scope.go:117] "RemoveContainer" containerID="28318e8db41e1065828de89c28b2e5dc4542ee2c4670f276aaa5477e57d60e6f" Sep 10 03:58:56.211012 ip-10-0-142-77 kubenswrapper[2581]: E0910 03:58:56.210993 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28318e8db41e1065828de89c28b2e5dc4542ee2c4670f276aaa5477e57d60e6f\": container with ID starting with 28318e8db41e1065828de89c28b2e5dc4542ee2c4670f276aaa5477e57d60e6f not found: ID does not exist" containerID="28318e8db41e1065828de89c28b2e5dc4542ee2c4670f276aaa5477e57d60e6f" Sep 10 03:58:56.211104 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:56.211020 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28318e8db41e1065828de89c28b2e5dc4542ee2c4670f276aaa5477e57d60e6f"} err="failed to get container status \"28318e8db41e1065828de89c28b2e5dc4542ee2c4670f276aaa5477e57d60e6f\": rpc error: code = NotFound desc = could not find container \"28318e8db41e1065828de89c28b2e5dc4542ee2c4670f276aaa5477e57d60e6f\": container with ID starting with 28318e8db41e1065828de89c28b2e5dc4542ee2c4670f276aaa5477e57d60e6f not found: ID does not exist" Sep 10 03:58:56.222009 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:56.221966 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cdfb499fc-592gv"] Sep 10 03:58:56.228485 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:56.228462 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-cdfb499fc-592gv"] Sep 10 03:58:56.941912 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:58:56.941879 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f28d5cf1-f8b6-4eb2-9b96-a74b97085b39" path="/var/lib/kubelet/pods/f28d5cf1-f8b6-4eb2-9b96-a74b97085b39/volumes" Sep 10 03:59:21.588484 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:21.588455 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv"] Sep 10 03:59:21.588865 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:21.588738 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f28d5cf1-f8b6-4eb2-9b96-a74b97085b39" containerName="console" Sep 10 03:59:21.588865 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:21.588749 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f28d5cf1-f8b6-4eb2-9b96-a74b97085b39" containerName="console" Sep 10 03:59:21.588865 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:21.588797 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f28d5cf1-f8b6-4eb2-9b96-a74b97085b39" containerName="console" Sep 10 03:59:21.592482 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:21.592459 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" Sep 10 03:59:21.596045 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:21.596023 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-prvxk\"" Sep 10 03:59:21.614185 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:21.614163 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv"] Sep 10 03:59:21.650683 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:21.650656 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00a43ad9-fe36-45f2-bdab-8b844574373d-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-j59lv\" (UID: \"00a43ad9-fe36-45f2-bdab-8b844574373d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" Sep 10 03:59:21.650683 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:21.650685 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00a43ad9-fe36-45f2-bdab-8b844574373d-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-j59lv\" (UID: \"00a43ad9-fe36-45f2-bdab-8b844574373d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" Sep 10 03:59:21.650879 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:21.650703 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00a43ad9-fe36-45f2-bdab-8b844574373d-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-j59lv\" (UID: \"00a43ad9-fe36-45f2-bdab-8b844574373d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" Sep 10 03:59:21.650879 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:21.650726 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00a43ad9-fe36-45f2-bdab-8b844574373d-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-j59lv\" (UID: \"00a43ad9-fe36-45f2-bdab-8b844574373d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" Sep 10 03:59:21.650879 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:21.650799 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00a43ad9-fe36-45f2-bdab-8b844574373d-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-j59lv\" (UID: \"00a43ad9-fe36-45f2-bdab-8b844574373d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" Sep 10 03:59:21.650879 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:21.650863 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00a43ad9-fe36-45f2-bdab-8b844574373d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-j59lv\" (UID: \"00a43ad9-fe36-45f2-bdab-8b844574373d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" Sep 10 03:59:21.651102 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:21.650902 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00a43ad9-fe36-45f2-bdab-8b844574373d-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-j59lv\" (UID: \"00a43ad9-fe36-45f2-bdab-8b844574373d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" Sep 10 03:59:21.651102 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:21.650958 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkll5\" (UniqueName: \"kubernetes.io/projected/00a43ad9-fe36-45f2-bdab-8b844574373d-kube-api-access-bkll5\") pod \"maas-default-gateway-openshift-default-5ccb968c66-j59lv\" (UID: \"00a43ad9-fe36-45f2-bdab-8b844574373d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" Sep 10 03:59:21.651102 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:21.651024 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00a43ad9-fe36-45f2-bdab-8b844574373d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-j59lv\" (UID: \"00a43ad9-fe36-45f2-bdab-8b844574373d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" Sep 10 03:59:21.751681 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:21.751653 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkll5\" (UniqueName: \"kubernetes.io/projected/00a43ad9-fe36-45f2-bdab-8b844574373d-kube-api-access-bkll5\") pod \"maas-default-gateway-openshift-default-5ccb968c66-j59lv\" (UID: \"00a43ad9-fe36-45f2-bdab-8b844574373d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" Sep 10 03:59:21.751830 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:21.751693 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00a43ad9-fe36-45f2-bdab-8b844574373d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-j59lv\" (UID: \"00a43ad9-fe36-45f2-bdab-8b844574373d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" Sep 10 03:59:21.751830 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:21.751738 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00a43ad9-fe36-45f2-bdab-8b844574373d-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-j59lv\" (UID: \"00a43ad9-fe36-45f2-bdab-8b844574373d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" Sep 10 03:59:21.751830 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:21.751754 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00a43ad9-fe36-45f2-bdab-8b844574373d-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-j59lv\" (UID: \"00a43ad9-fe36-45f2-bdab-8b844574373d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" Sep 10 03:59:21.751830 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:21.751778 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00a43ad9-fe36-45f2-bdab-8b844574373d-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-j59lv\" (UID: \"00a43ad9-fe36-45f2-bdab-8b844574373d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" Sep 10 03:59:21.752068 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:21.751908 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00a43ad9-fe36-45f2-bdab-8b844574373d-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-j59lv\" (UID: \"00a43ad9-fe36-45f2-bdab-8b844574373d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" Sep 10 03:59:21.752068 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:21.751946 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00a43ad9-fe36-45f2-bdab-8b844574373d-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-j59lv\" (UID: \"00a43ad9-fe36-45f2-bdab-8b844574373d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" Sep 10 03:59:21.752068 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:21.752036 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00a43ad9-fe36-45f2-bdab-8b844574373d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-j59lv\" (UID: \"00a43ad9-fe36-45f2-bdab-8b844574373d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" Sep 10 03:59:21.752068 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:21.752062 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00a43ad9-fe36-45f2-bdab-8b844574373d-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-j59lv\" (UID: \"00a43ad9-fe36-45f2-bdab-8b844574373d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" Sep 10 03:59:21.752267 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:21.752205 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00a43ad9-fe36-45f2-bdab-8b844574373d-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-j59lv\" (UID: \"00a43ad9-fe36-45f2-bdab-8b844574373d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" Sep 10 03:59:21.752267 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:21.752242 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00a43ad9-fe36-45f2-bdab-8b844574373d-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-j59lv\" (UID: \"00a43ad9-fe36-45f2-bdab-8b844574373d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" Sep 10 03:59:21.752390 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:21.752364 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00a43ad9-fe36-45f2-bdab-8b844574373d-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-j59lv\" (UID: \"00a43ad9-fe36-45f2-bdab-8b844574373d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" Sep 10 03:59:21.752390 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:21.752377 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00a43ad9-fe36-45f2-bdab-8b844574373d-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-j59lv\" (UID: \"00a43ad9-fe36-45f2-bdab-8b844574373d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" Sep 10 03:59:21.752606 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:21.752581 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00a43ad9-fe36-45f2-bdab-8b844574373d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-j59lv\" (UID: \"00a43ad9-fe36-45f2-bdab-8b844574373d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" Sep 10 03:59:21.754212 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:21.754187 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00a43ad9-fe36-45f2-bdab-8b844574373d-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-j59lv\" (UID: \"00a43ad9-fe36-45f2-bdab-8b844574373d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" Sep 10 03:59:21.754453 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:21.754436 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00a43ad9-fe36-45f2-bdab-8b844574373d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-j59lv\" (UID: \"00a43ad9-fe36-45f2-bdab-8b844574373d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" Sep 10 03:59:21.761920 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:21.761899 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkll5\" (UniqueName: \"kubernetes.io/projected/00a43ad9-fe36-45f2-bdab-8b844574373d-kube-api-access-bkll5\") pod \"maas-default-gateway-openshift-default-5ccb968c66-j59lv\" (UID: \"00a43ad9-fe36-45f2-bdab-8b844574373d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" Sep 10 03:59:21.762101 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:21.762086 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00a43ad9-fe36-45f2-bdab-8b844574373d-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-j59lv\" (UID: \"00a43ad9-fe36-45f2-bdab-8b844574373d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" Sep 10 03:59:21.902800 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:21.902719 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" Sep 10 03:59:22.174628 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:22.174546 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv"] Sep 10 03:59:22.177243 ip-10-0-142-77 kubenswrapper[2581]: W0910 03:59:22.177218 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00a43ad9_fe36_45f2_bdab_8b844574373d.slice/crio-825dd9ec9475014beb7c7889ac148f3ee837dc6b5f999666dc42eb6a42e9ef5c WatchSource:0}: Error finding container 825dd9ec9475014beb7c7889ac148f3ee837dc6b5f999666dc42eb6a42e9ef5c: Status 404 returned error can't find the container with id 825dd9ec9475014beb7c7889ac148f3ee837dc6b5f999666dc42eb6a42e9ef5c Sep 10 03:59:22.179222 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:22.179188 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 10 03:59:22.179301 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:22.179257 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 10 03:59:22.179301 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:22.179288 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 10 03:59:22.304893 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:22.304861 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" event={"ID":"00a43ad9-fe36-45f2-bdab-8b844574373d","Type":"ContainerStarted","Data":"74046ac2807df528df43af696267e0ec631b0834f1d2f057c4931435edab14a9"} Sep 10 03:59:22.305068 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:22.304916 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" event={"ID":"00a43ad9-fe36-45f2-bdab-8b844574373d","Type":"ContainerStarted","Data":"825dd9ec9475014beb7c7889ac148f3ee837dc6b5f999666dc42eb6a42e9ef5c"} Sep 10 03:59:22.324775 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:22.324725 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" podStartSLOduration=1.324708298 podStartE2EDuration="1.324708298s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 03:59:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 03:59:22.323159423 +0000 UTC m=+675.848538530" watchObservedRunningTime="2026-09-10 03:59:22.324708298 +0000 UTC m=+675.850087403" Sep 10 03:59:22.903065 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:22.903033 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" Sep 10 03:59:23.907685 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:23.907657 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" Sep 10 03:59:24.313741 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:24.313717 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" Sep 10 03:59:24.314613 ip-10-0-142-77 kubenswrapper[2581]: I0910 03:59:24.314596 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" Sep 10 04:00:58.702064 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:00:58.701955 2581 generic.go:356] "Generic (PLEG): container finished" podID="00a43ad9-fe36-45f2-bdab-8b844574373d" containerID="74046ac2807df528df43af696267e0ec631b0834f1d2f057c4931435edab14a9" exitCode=0 Sep 10 04:00:58.702064 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:00:58.702030 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" event={"ID":"00a43ad9-fe36-45f2-bdab-8b844574373d","Type":"ContainerDied","Data":"74046ac2807df528df43af696267e0ec631b0834f1d2f057c4931435edab14a9"} Sep 10 04:00:58.702561 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:00:58.702391 2581 scope.go:117] "RemoveContainer" containerID="74046ac2807df528df43af696267e0ec631b0834f1d2f057c4931435edab14a9" Sep 10 04:00:58.703322 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:00:58.703295 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 10 04:00:58.703413 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:00:58.703360 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 10 04:00:58.703413 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:00:58.703391 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 10 04:00:59.708242 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:00:59.708206 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" event={"ID":"00a43ad9-fe36-45f2-bdab-8b844574373d","Type":"ContainerStarted","Data":"e4325dc023991eff591c965a5b7cb4015233278b7924e0565340fdf40b2371c0"} Sep 10 04:00:59.903809 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:00:59.903774 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" Sep 10 04:00:59.904077 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:00:59.904058 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-j59lv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused" start-of-body= Sep 10 04:00:59.904131 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:00:59.904110 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" podUID="00a43ad9-fe36-45f2-bdab-8b844574373d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused" Sep 10 04:01:00.904163 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:00.904129 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-j59lv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused" start-of-body= Sep 10 04:01:00.904573 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:00.904199 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" podUID="00a43ad9-fe36-45f2-bdab-8b844574373d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused" Sep 10 04:01:01.903603 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:01.903566 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-j59lv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused" start-of-body= Sep 10 04:01:01.903781 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:01.903619 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" podUID="00a43ad9-fe36-45f2-bdab-8b844574373d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused" Sep 10 04:01:02.904124 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:02.904087 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-j59lv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused" start-of-body= Sep 10 04:01:02.904534 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:02.904147 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" podUID="00a43ad9-fe36-45f2-bdab-8b844574373d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused" Sep 10 04:01:03.903303 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:03.903267 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-j59lv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused" start-of-body= Sep 10 04:01:03.903486 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:03.903324 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" podUID="00a43ad9-fe36-45f2-bdab-8b844574373d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused" Sep 10 04:01:04.907802 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:04.907767 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" Sep 10 04:01:04.908227 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:04.907995 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" Sep 10 04:01:04.908699 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:04.908684 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-j59lv" Sep 10 04:01:20.941648 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:20.941620 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54"] Sep 10 04:01:20.944314 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:20.944296 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" Sep 10 04:01:20.947936 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:20.947912 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 10 04:01:20.948063 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:20.947952 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Sep 10 04:01:20.948063 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:20.947964 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4scvv\"" Sep 10 04:01:20.948063 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:20.947986 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 10 04:01:20.950183 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:20.950163 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54"] Sep 10 04:01:21.044183 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:21.044153 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c6ade3f-3c19-4b01-999f-f568494a937f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-859856b874-7lp54\" (UID: \"2c6ade3f-3c19-4b01-999f-f568494a937f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" Sep 10 04:01:21.044344 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:21.044190 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c6ade3f-3c19-4b01-999f-f568494a937f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-859856b874-7lp54\" (UID: \"2c6ade3f-3c19-4b01-999f-f568494a937f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" Sep 10 04:01:21.044344 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:21.044230 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c6ade3f-3c19-4b01-999f-f568494a937f-dshm\") pod \"e2e-distinct-2-simulated-kserve-859856b874-7lp54\" (UID: \"2c6ade3f-3c19-4b01-999f-f568494a937f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" Sep 10 04:01:21.044344 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:21.044250 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c6ade3f-3c19-4b01-999f-f568494a937f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-859856b874-7lp54\" (UID: \"2c6ade3f-3c19-4b01-999f-f568494a937f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" Sep 10 04:01:21.044344 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:21.044293 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c6ade3f-3c19-4b01-999f-f568494a937f-home\") pod \"e2e-distinct-2-simulated-kserve-859856b874-7lp54\" (UID: \"2c6ade3f-3c19-4b01-999f-f568494a937f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" Sep 10 04:01:21.044344 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:21.044311 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7h6f\" (UniqueName: \"kubernetes.io/projected/2c6ade3f-3c19-4b01-999f-f568494a937f-kube-api-access-w7h6f\") pod \"e2e-distinct-2-simulated-kserve-859856b874-7lp54\" (UID: \"2c6ade3f-3c19-4b01-999f-f568494a937f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" Sep 10 04:01:21.144958 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:21.144921 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c6ade3f-3c19-4b01-999f-f568494a937f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-859856b874-7lp54\" (UID: \"2c6ade3f-3c19-4b01-999f-f568494a937f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" Sep 10 04:01:21.144958 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:21.144959 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c6ade3f-3c19-4b01-999f-f568494a937f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-859856b874-7lp54\" (UID: \"2c6ade3f-3c19-4b01-999f-f568494a937f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" Sep 10 04:01:21.145220 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:21.145027 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c6ade3f-3c19-4b01-999f-f568494a937f-dshm\") pod \"e2e-distinct-2-simulated-kserve-859856b874-7lp54\" (UID: \"2c6ade3f-3c19-4b01-999f-f568494a937f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" Sep 10 04:01:21.145220 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:21.145048 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c6ade3f-3c19-4b01-999f-f568494a937f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-859856b874-7lp54\" (UID: \"2c6ade3f-3c19-4b01-999f-f568494a937f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" Sep 10 04:01:21.145220 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:21.145068 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c6ade3f-3c19-4b01-999f-f568494a937f-home\") pod \"e2e-distinct-2-simulated-kserve-859856b874-7lp54\" (UID: \"2c6ade3f-3c19-4b01-999f-f568494a937f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" Sep 10 04:01:21.145220 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:21.145093 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7h6f\" (UniqueName: \"kubernetes.io/projected/2c6ade3f-3c19-4b01-999f-f568494a937f-kube-api-access-w7h6f\") pod \"e2e-distinct-2-simulated-kserve-859856b874-7lp54\" (UID: \"2c6ade3f-3c19-4b01-999f-f568494a937f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" Sep 10 04:01:21.145573 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:21.145548 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c6ade3f-3c19-4b01-999f-f568494a937f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-859856b874-7lp54\" (UID: \"2c6ade3f-3c19-4b01-999f-f568494a937f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" Sep 10 04:01:21.145643 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:21.145583 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c6ade3f-3c19-4b01-999f-f568494a937f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-859856b874-7lp54\" (UID: \"2c6ade3f-3c19-4b01-999f-f568494a937f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" Sep 10 04:01:21.145643 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:21.145603 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c6ade3f-3c19-4b01-999f-f568494a937f-home\") pod \"e2e-distinct-2-simulated-kserve-859856b874-7lp54\" (UID: \"2c6ade3f-3c19-4b01-999f-f568494a937f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" Sep 10 04:01:21.147476 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:21.147454 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c6ade3f-3c19-4b01-999f-f568494a937f-dshm\") pod \"e2e-distinct-2-simulated-kserve-859856b874-7lp54\" (UID: \"2c6ade3f-3c19-4b01-999f-f568494a937f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" Sep 10 04:01:21.147636 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:21.147621 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c6ade3f-3c19-4b01-999f-f568494a937f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-859856b874-7lp54\" (UID: \"2c6ade3f-3c19-4b01-999f-f568494a937f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" Sep 10 04:01:21.153147 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:21.153121 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7h6f\" (UniqueName: \"kubernetes.io/projected/2c6ade3f-3c19-4b01-999f-f568494a937f-kube-api-access-w7h6f\") pod \"e2e-distinct-2-simulated-kserve-859856b874-7lp54\" (UID: \"2c6ade3f-3c19-4b01-999f-f568494a937f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" Sep 10 04:01:21.255856 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:21.255772 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" Sep 10 04:01:21.526005 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:21.525910 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54"] Sep 10 04:01:21.529635 ip-10-0-142-77 kubenswrapper[2581]: W0910 04:01:21.529606 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c6ade3f_3c19_4b01_999f_f568494a937f.slice/crio-72be8655e002006f253f07eca6344f8d3f141afa6efb12d34f11e1c1466d3cd3 WatchSource:0}: Error finding container 72be8655e002006f253f07eca6344f8d3f141afa6efb12d34f11e1c1466d3cd3: Status 404 returned error can't find the container with id 72be8655e002006f253f07eca6344f8d3f141afa6efb12d34f11e1c1466d3cd3 Sep 10 04:01:21.531233 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:21.531210 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 04:01:21.803275 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:21.803233 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" event={"ID":"2c6ade3f-3c19-4b01-999f-f568494a937f","Type":"ContainerStarted","Data":"72be8655e002006f253f07eca6344f8d3f141afa6efb12d34f11e1c1466d3cd3"} Sep 10 04:01:22.809802 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:22.809774 2581 generic.go:356] "Generic (PLEG): container finished" podID="2c6ade3f-3c19-4b01-999f-f568494a937f" containerID="9aea6003b9807b4bb73f3c5c185cf3c9df9ed1dd09d152bce17da1eaa23f1cb5" exitCode=0 Sep 10 04:01:22.810196 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:22.809841 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" event={"ID":"2c6ade3f-3c19-4b01-999f-f568494a937f","Type":"ContainerDied","Data":"9aea6003b9807b4bb73f3c5c185cf3c9df9ed1dd09d152bce17da1eaa23f1cb5"} Sep 10 04:01:22.810196 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:22.810100 2581 scope.go:117] "RemoveContainer" containerID="9aea6003b9807b4bb73f3c5c185cf3c9df9ed1dd09d152bce17da1eaa23f1cb5" Sep 10 04:01:23.816316 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:23.816281 2581 generic.go:356] "Generic (PLEG): container finished" podID="2c6ade3f-3c19-4b01-999f-f568494a937f" containerID="16ca89f7194cac9ac53ad35f710757f701b414004a7e904fd85c48b6187ac9e2" exitCode=0 Sep 10 04:01:23.816729 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:23.816330 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" event={"ID":"2c6ade3f-3c19-4b01-999f-f568494a937f","Type":"ContainerDied","Data":"16ca89f7194cac9ac53ad35f710757f701b414004a7e904fd85c48b6187ac9e2"} Sep 10 04:01:23.816729 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:23.816388 2581 scope.go:117] "RemoveContainer" containerID="9aea6003b9807b4bb73f3c5c185cf3c9df9ed1dd09d152bce17da1eaa23f1cb5" Sep 10 04:01:23.816729 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:23.816667 2581 scope.go:117] "RemoveContainer" containerID="16ca89f7194cac9ac53ad35f710757f701b414004a7e904fd85c48b6187ac9e2" Sep 10 04:01:23.816871 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:01:23.816853 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7lp54_llm(2c6ade3f-3c19-4b01-999f-f568494a937f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" podUID="2c6ade3f-3c19-4b01-999f-f568494a937f" Sep 10 04:01:24.822548 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:24.822518 2581 scope.go:117] "RemoveContainer" containerID="16ca89f7194cac9ac53ad35f710757f701b414004a7e904fd85c48b6187ac9e2" Sep 10 04:01:24.822923 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:01:24.822689 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7lp54_llm(2c6ade3f-3c19-4b01-999f-f568494a937f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" podUID="2c6ade3f-3c19-4b01-999f-f568494a937f" Sep 10 04:01:31.256347 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:31.256315 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" Sep 10 04:01:31.256826 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:31.256382 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" Sep 10 04:01:31.256826 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:31.256406 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" Sep 10 04:01:31.256826 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:31.256424 2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" Sep 10 04:01:31.256826 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:31.256757 2581 scope.go:117] "RemoveContainer" containerID="16ca89f7194cac9ac53ad35f710757f701b414004a7e904fd85c48b6187ac9e2" Sep 10 04:01:31.257054 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:01:31.256920 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7lp54_llm(2c6ade3f-3c19-4b01-999f-f568494a937f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" podUID="2c6ade3f-3c19-4b01-999f-f568494a937f" Sep 10 04:01:31.852458 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:31.852431 2581 scope.go:117] "RemoveContainer" containerID="16ca89f7194cac9ac53ad35f710757f701b414004a7e904fd85c48b6187ac9e2" Sep 10 04:01:31.852636 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:01:31.852617 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7lp54_llm(2c6ade3f-3c19-4b01-999f-f568494a937f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" podUID="2c6ade3f-3c19-4b01-999f-f568494a937f" Sep 10 04:01:37.736314 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:37.736266 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh"] Sep 10 04:01:37.739056 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:37.739033 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" Sep 10 04:01:37.741669 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:37.741647 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Sep 10 04:01:37.753017 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:37.750493 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh"] Sep 10 04:01:37.778947 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:37.778921 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57h4n\" (UniqueName: \"kubernetes.io/projected/4dc064fe-8790-42ff-8c51-a4dce6cd3731-kube-api-access-57h4n\") pod \"e2e-distinct-simulated-kserve-6685d86975-7zgvh\" (UID: \"4dc064fe-8790-42ff-8c51-a4dce6cd3731\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" Sep 10 04:01:37.779073 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:37.779008 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4dc064fe-8790-42ff-8c51-a4dce6cd3731-tls-certs\") pod \"e2e-distinct-simulated-kserve-6685d86975-7zgvh\" (UID: \"4dc064fe-8790-42ff-8c51-a4dce6cd3731\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" Sep 10 04:01:37.779073 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:37.779029 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4dc064fe-8790-42ff-8c51-a4dce6cd3731-dshm\") pod \"e2e-distinct-simulated-kserve-6685d86975-7zgvh\" (UID: \"4dc064fe-8790-42ff-8c51-a4dce6cd3731\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" Sep 10 04:01:37.779147 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:37.779073 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4dc064fe-8790-42ff-8c51-a4dce6cd3731-tmp-dir\") pod \"e2e-distinct-simulated-kserve-6685d86975-7zgvh\" (UID: \"4dc064fe-8790-42ff-8c51-a4dce6cd3731\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" Sep 10 04:01:37.779147 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:37.779102 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4dc064fe-8790-42ff-8c51-a4dce6cd3731-model-cache\") pod \"e2e-distinct-simulated-kserve-6685d86975-7zgvh\" (UID: \"4dc064fe-8790-42ff-8c51-a4dce6cd3731\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" Sep 10 04:01:37.779147 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:37.779116 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4dc064fe-8790-42ff-8c51-a4dce6cd3731-home\") pod \"e2e-distinct-simulated-kserve-6685d86975-7zgvh\" (UID: \"4dc064fe-8790-42ff-8c51-a4dce6cd3731\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" Sep 10 04:01:37.880222 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:37.880190 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4dc064fe-8790-42ff-8c51-a4dce6cd3731-tls-certs\") pod \"e2e-distinct-simulated-kserve-6685d86975-7zgvh\" (UID: \"4dc064fe-8790-42ff-8c51-a4dce6cd3731\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" Sep 10 04:01:37.880222 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:37.880222 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4dc064fe-8790-42ff-8c51-a4dce6cd3731-dshm\") pod \"e2e-distinct-simulated-kserve-6685d86975-7zgvh\" (UID: \"4dc064fe-8790-42ff-8c51-a4dce6cd3731\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" Sep 10 04:01:37.880407 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:37.880262 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4dc064fe-8790-42ff-8c51-a4dce6cd3731-tmp-dir\") pod \"e2e-distinct-simulated-kserve-6685d86975-7zgvh\" (UID: \"4dc064fe-8790-42ff-8c51-a4dce6cd3731\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" Sep 10 04:01:37.880407 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:37.880355 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4dc064fe-8790-42ff-8c51-a4dce6cd3731-model-cache\") pod \"e2e-distinct-simulated-kserve-6685d86975-7zgvh\" (UID: \"4dc064fe-8790-42ff-8c51-a4dce6cd3731\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" Sep 10 04:01:37.880407 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:37.880386 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4dc064fe-8790-42ff-8c51-a4dce6cd3731-home\") pod \"e2e-distinct-simulated-kserve-6685d86975-7zgvh\" (UID: \"4dc064fe-8790-42ff-8c51-a4dce6cd3731\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" Sep 10 04:01:37.880532 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:37.880416 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57h4n\" (UniqueName: \"kubernetes.io/projected/4dc064fe-8790-42ff-8c51-a4dce6cd3731-kube-api-access-57h4n\") pod \"e2e-distinct-simulated-kserve-6685d86975-7zgvh\" (UID: \"4dc064fe-8790-42ff-8c51-a4dce6cd3731\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" Sep 10 04:01:37.880690 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:37.880662 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4dc064fe-8790-42ff-8c51-a4dce6cd3731-tmp-dir\") pod \"e2e-distinct-simulated-kserve-6685d86975-7zgvh\" (UID: \"4dc064fe-8790-42ff-8c51-a4dce6cd3731\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" Sep 10 04:01:37.880794 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:37.880671 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4dc064fe-8790-42ff-8c51-a4dce6cd3731-model-cache\") pod \"e2e-distinct-simulated-kserve-6685d86975-7zgvh\" (UID: \"4dc064fe-8790-42ff-8c51-a4dce6cd3731\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" Sep 10 04:01:37.880794 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:37.880736 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4dc064fe-8790-42ff-8c51-a4dce6cd3731-home\") pod \"e2e-distinct-simulated-kserve-6685d86975-7zgvh\" (UID: \"4dc064fe-8790-42ff-8c51-a4dce6cd3731\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" Sep 10 04:01:37.882692 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:37.882666 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4dc064fe-8790-42ff-8c51-a4dce6cd3731-dshm\") pod \"e2e-distinct-simulated-kserve-6685d86975-7zgvh\" (UID: \"4dc064fe-8790-42ff-8c51-a4dce6cd3731\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" Sep 10 04:01:37.882903 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:37.882885 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4dc064fe-8790-42ff-8c51-a4dce6cd3731-tls-certs\") pod \"e2e-distinct-simulated-kserve-6685d86975-7zgvh\" (UID: \"4dc064fe-8790-42ff-8c51-a4dce6cd3731\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" Sep 10 04:01:37.890129 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:37.890106 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57h4n\" (UniqueName: \"kubernetes.io/projected/4dc064fe-8790-42ff-8c51-a4dce6cd3731-kube-api-access-57h4n\") pod \"e2e-distinct-simulated-kserve-6685d86975-7zgvh\" (UID: \"4dc064fe-8790-42ff-8c51-a4dce6cd3731\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" Sep 10 04:01:38.051866 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:38.051828 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" Sep 10 04:01:38.318955 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:38.318934 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh"] Sep 10 04:01:38.320625 ip-10-0-142-77 kubenswrapper[2581]: W0910 04:01:38.320597 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dc064fe_8790_42ff_8c51_a4dce6cd3731.slice/crio-4476604e41d6e30f717cb6530612b7b44bd00d278b5287e8b44f65d83502cf8f WatchSource:0}: Error finding container 4476604e41d6e30f717cb6530612b7b44bd00d278b5287e8b44f65d83502cf8f: Status 404 returned error can't find the container with id 4476604e41d6e30f717cb6530612b7b44bd00d278b5287e8b44f65d83502cf8f Sep 10 04:01:38.881427 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:38.881340 2581 generic.go:356] "Generic (PLEG): container finished" podID="4dc064fe-8790-42ff-8c51-a4dce6cd3731" containerID="1a47258cffae78e156d9152177fbaa30b8524b246191e82f03ce2d0a18326cf7" exitCode=0 Sep 10 04:01:38.881427 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:38.881410 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" event={"ID":"4dc064fe-8790-42ff-8c51-a4dce6cd3731","Type":"ContainerDied","Data":"1a47258cffae78e156d9152177fbaa30b8524b246191e82f03ce2d0a18326cf7"} Sep 10 04:01:38.881828 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:38.881447 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" event={"ID":"4dc064fe-8790-42ff-8c51-a4dce6cd3731","Type":"ContainerStarted","Data":"4476604e41d6e30f717cb6530612b7b44bd00d278b5287e8b44f65d83502cf8f"} Sep 10 04:01:38.881828 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:38.881661 2581 scope.go:117] "RemoveContainer" containerID="1a47258cffae78e156d9152177fbaa30b8524b246191e82f03ce2d0a18326cf7" Sep 10 04:01:39.887874 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:39.887844 2581 generic.go:356] "Generic (PLEG): container finished" podID="4dc064fe-8790-42ff-8c51-a4dce6cd3731" containerID="b5a5ddaa01edc843e190993b50fd84f6522073797c4a818268006afdbcac43c5" exitCode=0 Sep 10 04:01:39.888328 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:39.887987 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" event={"ID":"4dc064fe-8790-42ff-8c51-a4dce6cd3731","Type":"ContainerDied","Data":"b5a5ddaa01edc843e190993b50fd84f6522073797c4a818268006afdbcac43c5"} Sep 10 04:01:39.888328 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:39.888049 2581 scope.go:117] "RemoveContainer" containerID="1a47258cffae78e156d9152177fbaa30b8524b246191e82f03ce2d0a18326cf7" Sep 10 04:01:39.888415 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:39.888337 2581 scope.go:117] "RemoveContainer" containerID="b5a5ddaa01edc843e190993b50fd84f6522073797c4a818268006afdbcac43c5" Sep 10 04:01:39.888552 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:01:39.888533 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7zgvh_llm(4dc064fe-8790-42ff-8c51-a4dce6cd3731)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" podUID="4dc064fe-8790-42ff-8c51-a4dce6cd3731" Sep 10 04:01:40.894316 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:40.894291 2581 scope.go:117] "RemoveContainer" containerID="b5a5ddaa01edc843e190993b50fd84f6522073797c4a818268006afdbcac43c5" Sep 10 04:01:40.894691 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:01:40.894465 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7zgvh_llm(4dc064fe-8790-42ff-8c51-a4dce6cd3731)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" podUID="4dc064fe-8790-42ff-8c51-a4dce6cd3731" Sep 10 04:01:42.937670 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:42.937638 2581 scope.go:117] "RemoveContainer" containerID="16ca89f7194cac9ac53ad35f710757f701b414004a7e904fd85c48b6187ac9e2" Sep 10 04:01:43.912710 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:43.912681 2581 generic.go:356] "Generic (PLEG): container finished" podID="2c6ade3f-3c19-4b01-999f-f568494a937f" containerID="9ed66ecdd0d5196b86a7b7b26e9f6a6032893567c3f31ba3376bee10be137d48" exitCode=0 Sep 10 04:01:43.912880 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:43.912755 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" event={"ID":"2c6ade3f-3c19-4b01-999f-f568494a937f","Type":"ContainerDied","Data":"9ed66ecdd0d5196b86a7b7b26e9f6a6032893567c3f31ba3376bee10be137d48"} Sep 10 04:01:43.912880 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:43.912832 2581 scope.go:117] "RemoveContainer" containerID="16ca89f7194cac9ac53ad35f710757f701b414004a7e904fd85c48b6187ac9e2" Sep 10 04:01:43.913192 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:43.913176 2581 scope.go:117] "RemoveContainer" containerID="9ed66ecdd0d5196b86a7b7b26e9f6a6032893567c3f31ba3376bee10be137d48" Sep 10 04:01:43.913379 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:01:43.913361 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7lp54_llm(2c6ade3f-3c19-4b01-999f-f568494a937f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" podUID="2c6ade3f-3c19-4b01-999f-f568494a937f" Sep 10 04:01:48.052316 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:48.052270 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" Sep 10 04:01:48.052316 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:48.052334 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" Sep 10 04:01:48.052770 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:48.052351 2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" Sep 10 04:01:48.052770 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:48.052367 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" Sep 10 04:01:48.052770 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:48.052729 2581 scope.go:117] "RemoveContainer" containerID="b5a5ddaa01edc843e190993b50fd84f6522073797c4a818268006afdbcac43c5" Sep 10 04:01:48.052908 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:01:48.052890 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7zgvh_llm(4dc064fe-8790-42ff-8c51-a4dce6cd3731)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" podUID="4dc064fe-8790-42ff-8c51-a4dce6cd3731" Sep 10 04:01:51.256225 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:51.256186 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" Sep 10 04:01:51.256640 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:51.256276 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" Sep 10 04:01:51.256640 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:51.256293 2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" Sep 10 04:01:51.256640 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:51.256308 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" Sep 10 04:01:51.256748 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:01:51.256653 2581 scope.go:117] "RemoveContainer" containerID="9ed66ecdd0d5196b86a7b7b26e9f6a6032893567c3f31ba3376bee10be137d48" Sep 10 04:01:51.256908 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:01:51.256882 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7lp54_llm(2c6ade3f-3c19-4b01-999f-f568494a937f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" podUID="2c6ade3f-3c19-4b01-999f-f568494a937f" Sep 10 04:02:02.936964 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:02.936928 2581 scope.go:117] "RemoveContainer" containerID="b5a5ddaa01edc843e190993b50fd84f6522073797c4a818268006afdbcac43c5" Sep 10 04:02:03.997906 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:03.997873 2581 generic.go:356] "Generic (PLEG): container finished" podID="4dc064fe-8790-42ff-8c51-a4dce6cd3731" containerID="513a9d59b31c7e02dc46152c387fc0b3fedc6fc40f182c0e11316084031af58e" exitCode=0 Sep 10 04:02:03.998347 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:03.997928 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" event={"ID":"4dc064fe-8790-42ff-8c51-a4dce6cd3731","Type":"ContainerDied","Data":"513a9d59b31c7e02dc46152c387fc0b3fedc6fc40f182c0e11316084031af58e"} Sep 10 04:02:03.998347 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:03.997990 2581 scope.go:117] "RemoveContainer" containerID="b5a5ddaa01edc843e190993b50fd84f6522073797c4a818268006afdbcac43c5" Sep 10 04:02:03.998435 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:03.998354 2581 scope.go:117] "RemoveContainer" containerID="513a9d59b31c7e02dc46152c387fc0b3fedc6fc40f182c0e11316084031af58e" Sep 10 04:02:03.998575 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:02:03.998555 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7zgvh_llm(4dc064fe-8790-42ff-8c51-a4dce6cd3731)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" podUID="4dc064fe-8790-42ff-8c51-a4dce6cd3731" Sep 10 04:02:05.937213 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:05.937176 2581 scope.go:117] "RemoveContainer" containerID="9ed66ecdd0d5196b86a7b7b26e9f6a6032893567c3f31ba3376bee10be137d48" Sep 10 04:02:07.014268 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:07.014238 2581 generic.go:356] "Generic (PLEG): container finished" podID="2c6ade3f-3c19-4b01-999f-f568494a937f" containerID="a7d084b250c6ece1aaab56e5675dff9c396e8844998352ddaf52a6f8f4270a56" exitCode=0 Sep 10 04:02:07.014635 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:07.014287 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" event={"ID":"2c6ade3f-3c19-4b01-999f-f568494a937f","Type":"ContainerDied","Data":"a7d084b250c6ece1aaab56e5675dff9c396e8844998352ddaf52a6f8f4270a56"} Sep 10 04:02:07.014635 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:07.014332 2581 scope.go:117] "RemoveContainer" containerID="9ed66ecdd0d5196b86a7b7b26e9f6a6032893567c3f31ba3376bee10be137d48" Sep 10 04:02:07.014788 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:07.014655 2581 scope.go:117] "RemoveContainer" containerID="a7d084b250c6ece1aaab56e5675dff9c396e8844998352ddaf52a6f8f4270a56" Sep 10 04:02:07.014847 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:02:07.014829 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7lp54_llm(2c6ade3f-3c19-4b01-999f-f568494a937f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" podUID="2c6ade3f-3c19-4b01-999f-f568494a937f" Sep 10 04:02:08.052153 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:08.052120 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" Sep 10 04:02:08.052536 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:08.052199 2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" Sep 10 04:02:08.052536 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:08.052215 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" Sep 10 04:02:08.052536 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:08.052227 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" Sep 10 04:02:08.052640 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:08.052619 2581 scope.go:117] "RemoveContainer" containerID="513a9d59b31c7e02dc46152c387fc0b3fedc6fc40f182c0e11316084031af58e" Sep 10 04:02:08.052833 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:02:08.052809 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7zgvh_llm(4dc064fe-8790-42ff-8c51-a4dce6cd3731)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" podUID="4dc064fe-8790-42ff-8c51-a4dce6cd3731" Sep 10 04:02:11.256114 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:11.256080 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" Sep 10 04:02:11.256484 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:11.256142 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" Sep 10 04:02:11.256484 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:11.256158 2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" Sep 10 04:02:11.256484 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:11.256171 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" Sep 10 04:02:11.256602 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:11.256511 2581 scope.go:117] "RemoveContainer" containerID="a7d084b250c6ece1aaab56e5675dff9c396e8844998352ddaf52a6f8f4270a56" Sep 10 04:02:11.256707 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:02:11.256687 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7lp54_llm(2c6ade3f-3c19-4b01-999f-f568494a937f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" podUID="2c6ade3f-3c19-4b01-999f-f568494a937f" Sep 10 04:02:21.937554 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:21.937521 2581 scope.go:117] "RemoveContainer" containerID="513a9d59b31c7e02dc46152c387fc0b3fedc6fc40f182c0e11316084031af58e" Sep 10 04:02:21.937554 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:21.937552 2581 scope.go:117] "RemoveContainer" containerID="a7d084b250c6ece1aaab56e5675dff9c396e8844998352ddaf52a6f8f4270a56" Sep 10 04:02:21.937991 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:02:21.937742 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7zgvh_llm(4dc064fe-8790-42ff-8c51-a4dce6cd3731)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" podUID="4dc064fe-8790-42ff-8c51-a4dce6cd3731" Sep 10 04:02:21.937991 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:02:21.937777 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7lp54_llm(2c6ade3f-3c19-4b01-999f-f568494a937f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" podUID="2c6ade3f-3c19-4b01-999f-f568494a937f" Sep 10 04:02:33.937399 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:33.937361 2581 scope.go:117] "RemoveContainer" containerID="513a9d59b31c7e02dc46152c387fc0b3fedc6fc40f182c0e11316084031af58e" Sep 10 04:02:35.128809 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:35.128776 2581 generic.go:356] "Generic (PLEG): container finished" podID="4dc064fe-8790-42ff-8c51-a4dce6cd3731" containerID="a4aec8ea267716d5a0c76a7336a6b63bc7e9fcf83b23510b7ae7bced44f4f3a0" exitCode=0 Sep 10 04:02:35.129230 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:35.128835 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" event={"ID":"4dc064fe-8790-42ff-8c51-a4dce6cd3731","Type":"ContainerDied","Data":"a4aec8ea267716d5a0c76a7336a6b63bc7e9fcf83b23510b7ae7bced44f4f3a0"} Sep 10 04:02:35.129230 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:35.128888 2581 scope.go:117] "RemoveContainer" containerID="513a9d59b31c7e02dc46152c387fc0b3fedc6fc40f182c0e11316084031af58e" Sep 10 04:02:35.129330 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:35.129279 2581 scope.go:117] "RemoveContainer" containerID="a4aec8ea267716d5a0c76a7336a6b63bc7e9fcf83b23510b7ae7bced44f4f3a0" Sep 10 04:02:35.129479 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:02:35.129456 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7zgvh_llm(4dc064fe-8790-42ff-8c51-a4dce6cd3731)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" podUID="4dc064fe-8790-42ff-8c51-a4dce6cd3731" Sep 10 04:02:35.937463 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:35.937428 2581 scope.go:117] "RemoveContainer" containerID="a7d084b250c6ece1aaab56e5675dff9c396e8844998352ddaf52a6f8f4270a56" Sep 10 04:02:35.937636 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:02:35.937607 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7lp54_llm(2c6ade3f-3c19-4b01-999f-f568494a937f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" podUID="2c6ade3f-3c19-4b01-999f-f568494a937f" Sep 10 04:02:38.052664 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:38.052625 2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" Sep 10 04:02:38.053073 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:38.052685 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" Sep 10 04:02:38.053073 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:38.052702 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" Sep 10 04:02:38.053073 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:38.052717 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" Sep 10 04:02:38.053170 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:38.053073 2581 scope.go:117] "RemoveContainer" containerID="a4aec8ea267716d5a0c76a7336a6b63bc7e9fcf83b23510b7ae7bced44f4f3a0" Sep 10 04:02:38.053262 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:02:38.053242 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7zgvh_llm(4dc064fe-8790-42ff-8c51-a4dce6cd3731)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" podUID="4dc064fe-8790-42ff-8c51-a4dce6cd3731" Sep 10 04:02:48.937656 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:48.937619 2581 scope.go:117] "RemoveContainer" containerID="a7d084b250c6ece1aaab56e5675dff9c396e8844998352ddaf52a6f8f4270a56" Sep 10 04:02:49.193604 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:49.193551 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" event={"ID":"2c6ade3f-3c19-4b01-999f-f568494a937f","Type":"ContainerStarted","Data":"c23f20e1acf0455b3a05453678c28918590accb834a172a2863343f7be6f88c8"} Sep 10 04:02:49.212456 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:49.212401 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" podStartSLOduration=87.27399493 podStartE2EDuration="1m29.212379866s" totalImagesPullingTime="1.938384936s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 04:01:20 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:02:49.210853609 +0000 UTC m=+882.736232710" watchObservedRunningTime="2026-09-10 04:02:49.212379866 +0000 UTC m=+882.737758964" Sep 10 04:02:50.200128 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:50.200096 2581 generic.go:356] "Generic (PLEG): container finished" podID="2c6ade3f-3c19-4b01-999f-f568494a937f" containerID="c23f20e1acf0455b3a05453678c28918590accb834a172a2863343f7be6f88c8" exitCode=0 Sep 10 04:02:50.200567 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:50.200160 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" event={"ID":"2c6ade3f-3c19-4b01-999f-f568494a937f","Type":"ContainerDied","Data":"c23f20e1acf0455b3a05453678c28918590accb834a172a2863343f7be6f88c8"} Sep 10 04:02:50.200567 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:50.200207 2581 scope.go:117] "RemoveContainer" containerID="a7d084b250c6ece1aaab56e5675dff9c396e8844998352ddaf52a6f8f4270a56" Sep 10 04:02:50.200567 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:50.200546 2581 scope.go:117] "RemoveContainer" containerID="c23f20e1acf0455b3a05453678c28918590accb834a172a2863343f7be6f88c8" Sep 10 04:02:50.200764 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:02:50.200744 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7lp54_llm(2c6ade3f-3c19-4b01-999f-f568494a937f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" podUID="2c6ade3f-3c19-4b01-999f-f568494a937f" Sep 10 04:02:51.256028 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:51.255988 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" Sep 10 04:02:51.256433 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:51.256051 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" Sep 10 04:02:51.256433 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:51.256066 2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" Sep 10 04:02:51.256433 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:51.256079 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" Sep 10 04:02:51.256433 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:51.256391 2581 scope.go:117] "RemoveContainer" containerID="c23f20e1acf0455b3a05453678c28918590accb834a172a2863343f7be6f88c8" Sep 10 04:02:51.256616 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:02:51.256558 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7lp54_llm(2c6ade3f-3c19-4b01-999f-f568494a937f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" podUID="2c6ade3f-3c19-4b01-999f-f568494a937f" Sep 10 04:02:52.942364 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:02:52.942340 2581 scope.go:117] "RemoveContainer" containerID="a4aec8ea267716d5a0c76a7336a6b63bc7e9fcf83b23510b7ae7bced44f4f3a0" Sep 10 04:02:52.942748 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:02:52.942518 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7zgvh_llm(4dc064fe-8790-42ff-8c51-a4dce6cd3731)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" podUID="4dc064fe-8790-42ff-8c51-a4dce6cd3731" Sep 10 04:03:03.937476 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:03:03.937395 2581 scope.go:117] "RemoveContainer" containerID="c23f20e1acf0455b3a05453678c28918590accb834a172a2863343f7be6f88c8" Sep 10 04:03:03.938024 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:03:03.937608 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7lp54_llm(2c6ade3f-3c19-4b01-999f-f568494a937f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" podUID="2c6ade3f-3c19-4b01-999f-f568494a937f" Sep 10 04:03:07.937086 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:03:07.937051 2581 scope.go:117] "RemoveContainer" containerID="a4aec8ea267716d5a0c76a7336a6b63bc7e9fcf83b23510b7ae7bced44f4f3a0" Sep 10 04:03:07.937455 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:03:07.937281 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7zgvh_llm(4dc064fe-8790-42ff-8c51-a4dce6cd3731)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" podUID="4dc064fe-8790-42ff-8c51-a4dce6cd3731" Sep 10 04:03:17.937251 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:03:17.937218 2581 scope.go:117] "RemoveContainer" containerID="c23f20e1acf0455b3a05453678c28918590accb834a172a2863343f7be6f88c8" Sep 10 04:03:17.937706 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:03:17.937397 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7lp54_llm(2c6ade3f-3c19-4b01-999f-f568494a937f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" podUID="2c6ade3f-3c19-4b01-999f-f568494a937f" Sep 10 04:03:20.937482 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:03:20.937448 2581 scope.go:117] "RemoveContainer" containerID="a4aec8ea267716d5a0c76a7336a6b63bc7e9fcf83b23510b7ae7bced44f4f3a0" Sep 10 04:03:21.342674 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:03:21.342644 2581 generic.go:356] "Generic (PLEG): container finished" podID="4dc064fe-8790-42ff-8c51-a4dce6cd3731" containerID="73be814c62ef661f95e47e455ce0366e6aac65f49910616aa57caac0d9b60fed" exitCode=0 Sep 10 04:03:21.342823 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:03:21.342713 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" event={"ID":"4dc064fe-8790-42ff-8c51-a4dce6cd3731","Type":"ContainerDied","Data":"73be814c62ef661f95e47e455ce0366e6aac65f49910616aa57caac0d9b60fed"} Sep 10 04:03:21.342823 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:03:21.342778 2581 scope.go:117] "RemoveContainer" containerID="a4aec8ea267716d5a0c76a7336a6b63bc7e9fcf83b23510b7ae7bced44f4f3a0" Sep 10 04:03:21.343194 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:03:21.343180 2581 scope.go:117] "RemoveContainer" containerID="73be814c62ef661f95e47e455ce0366e6aac65f49910616aa57caac0d9b60fed" Sep 10 04:03:21.343390 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:03:21.343369 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7zgvh_llm(4dc064fe-8790-42ff-8c51-a4dce6cd3731)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" podUID="4dc064fe-8790-42ff-8c51-a4dce6cd3731" Sep 10 04:03:28.052621 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:03:28.052589 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" Sep 10 04:03:28.053150 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:03:28.052641 2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" Sep 10 04:03:28.053150 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:03:28.052688 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" Sep 10 04:03:28.053150 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:03:28.052733 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" Sep 10 04:03:28.053150 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:03:28.053080 2581 scope.go:117] "RemoveContainer" containerID="73be814c62ef661f95e47e455ce0366e6aac65f49910616aa57caac0d9b60fed" Sep 10 04:03:28.053299 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:03:28.053285 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7zgvh_llm(4dc064fe-8790-42ff-8c51-a4dce6cd3731)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" podUID="4dc064fe-8790-42ff-8c51-a4dce6cd3731" Sep 10 04:03:28.374319 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:03:28.374244 2581 scope.go:117] "RemoveContainer" containerID="73be814c62ef661f95e47e455ce0366e6aac65f49910616aa57caac0d9b60fed" Sep 10 04:03:28.374460 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:03:28.374403 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7zgvh_llm(4dc064fe-8790-42ff-8c51-a4dce6cd3731)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" podUID="4dc064fe-8790-42ff-8c51-a4dce6cd3731" Sep 10 04:03:30.941920 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:03:30.941884 2581 scope.go:117] "RemoveContainer" containerID="c23f20e1acf0455b3a05453678c28918590accb834a172a2863343f7be6f88c8" Sep 10 04:03:30.942403 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:03:30.942125 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7lp54_llm(2c6ade3f-3c19-4b01-999f-f568494a937f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" podUID="2c6ade3f-3c19-4b01-999f-f568494a937f" Sep 10 04:03:41.936952 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:03:41.936911 2581 scope.go:117] "RemoveContainer" containerID="73be814c62ef661f95e47e455ce0366e6aac65f49910616aa57caac0d9b60fed" Sep 10 04:03:41.937584 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:03:41.937127 2581 scope.go:117] "RemoveContainer" containerID="c23f20e1acf0455b3a05453678c28918590accb834a172a2863343f7be6f88c8" Sep 10 04:03:41.937584 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:03:41.937128 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7zgvh_llm(4dc064fe-8790-42ff-8c51-a4dce6cd3731)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" podUID="4dc064fe-8790-42ff-8c51-a4dce6cd3731" Sep 10 04:03:41.937584 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:03:41.937308 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7lp54_llm(2c6ade3f-3c19-4b01-999f-f568494a937f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" podUID="2c6ade3f-3c19-4b01-999f-f568494a937f" Sep 10 04:03:53.937256 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:03:53.937224 2581 scope.go:117] "RemoveContainer" containerID="73be814c62ef661f95e47e455ce0366e6aac65f49910616aa57caac0d9b60fed" Sep 10 04:03:53.937717 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:03:53.937315 2581 scope.go:117] "RemoveContainer" containerID="c23f20e1acf0455b3a05453678c28918590accb834a172a2863343f7be6f88c8" Sep 10 04:03:53.937717 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:03:53.937402 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7zgvh_llm(4dc064fe-8790-42ff-8c51-a4dce6cd3731)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" podUID="4dc064fe-8790-42ff-8c51-a4dce6cd3731" Sep 10 04:03:53.937717 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:03:53.937488 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7lp54_llm(2c6ade3f-3c19-4b01-999f-f568494a937f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" podUID="2c6ade3f-3c19-4b01-999f-f568494a937f" Sep 10 04:04:04.936867 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:04:04.936831 2581 scope.go:117] "RemoveContainer" containerID="73be814c62ef661f95e47e455ce0366e6aac65f49910616aa57caac0d9b60fed" Sep 10 04:04:04.937362 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:04:04.937031 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7zgvh_llm(4dc064fe-8790-42ff-8c51-a4dce6cd3731)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" podUID="4dc064fe-8790-42ff-8c51-a4dce6cd3731" Sep 10 04:04:06.938685 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:04:06.938656 2581 scope.go:117] "RemoveContainer" containerID="c23f20e1acf0455b3a05453678c28918590accb834a172a2863343f7be6f88c8" Sep 10 04:04:06.939074 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:04:06.938919 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7lp54_llm(2c6ade3f-3c19-4b01-999f-f568494a937f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" podUID="2c6ade3f-3c19-4b01-999f-f568494a937f" Sep 10 04:04:15.937038 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:04:15.937005 2581 scope.go:117] "RemoveContainer" containerID="73be814c62ef661f95e47e455ce0366e6aac65f49910616aa57caac0d9b60fed" Sep 10 04:04:15.937521 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:04:15.937174 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7zgvh_llm(4dc064fe-8790-42ff-8c51-a4dce6cd3731)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" podUID="4dc064fe-8790-42ff-8c51-a4dce6cd3731" Sep 10 04:04:18.937297 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:04:18.937264 2581 scope.go:117] "RemoveContainer" containerID="c23f20e1acf0455b3a05453678c28918590accb834a172a2863343f7be6f88c8" Sep 10 04:04:19.581892 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:04:19.581860 2581 generic.go:356] "Generic (PLEG): container finished" podID="2c6ade3f-3c19-4b01-999f-f568494a937f" containerID="3cf054fbd398ad0be58323eb1c8032af151525e64ac1f1bc9d3e7ea2f3335604" exitCode=0 Sep 10 04:04:19.582107 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:04:19.581939 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" event={"ID":"2c6ade3f-3c19-4b01-999f-f568494a937f","Type":"ContainerDied","Data":"3cf054fbd398ad0be58323eb1c8032af151525e64ac1f1bc9d3e7ea2f3335604"} Sep 10 04:04:19.582107 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:04:19.582034 2581 scope.go:117] "RemoveContainer" containerID="c23f20e1acf0455b3a05453678c28918590accb834a172a2863343f7be6f88c8" Sep 10 04:04:19.582401 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:04:19.582387 2581 scope.go:117] "RemoveContainer" containerID="3cf054fbd398ad0be58323eb1c8032af151525e64ac1f1bc9d3e7ea2f3335604" Sep 10 04:04:19.582601 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:04:19.582581 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7lp54_llm(2c6ade3f-3c19-4b01-999f-f568494a937f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" podUID="2c6ade3f-3c19-4b01-999f-f568494a937f" Sep 10 04:04:21.256114 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:04:21.256076 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" Sep 10 04:04:21.256563 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:04:21.256151 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" Sep 10 04:04:21.256563 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:04:21.256171 2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" Sep 10 04:04:21.256563 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:04:21.256190 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" Sep 10 04:04:21.256563 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:04:21.256520 2581 scope.go:117] "RemoveContainer" containerID="3cf054fbd398ad0be58323eb1c8032af151525e64ac1f1bc9d3e7ea2f3335604" Sep 10 04:04:21.256707 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:04:21.256687 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7lp54_llm(2c6ade3f-3c19-4b01-999f-f568494a937f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" podUID="2c6ade3f-3c19-4b01-999f-f568494a937f" Sep 10 04:04:26.938743 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:04:26.938708 2581 scope.go:117] "RemoveContainer" containerID="73be814c62ef661f95e47e455ce0366e6aac65f49910616aa57caac0d9b60fed" Sep 10 04:04:26.939142 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:04:26.938876 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7zgvh_llm(4dc064fe-8790-42ff-8c51-a4dce6cd3731)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" podUID="4dc064fe-8790-42ff-8c51-a4dce6cd3731" Sep 10 04:04:35.937337 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:04:35.937304 2581 scope.go:117] "RemoveContainer" containerID="3cf054fbd398ad0be58323eb1c8032af151525e64ac1f1bc9d3e7ea2f3335604" Sep 10 04:04:35.937715 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:04:35.937533 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7lp54_llm(2c6ade3f-3c19-4b01-999f-f568494a937f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" podUID="2c6ade3f-3c19-4b01-999f-f568494a937f" Sep 10 04:04:38.936871 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:04:38.936834 2581 scope.go:117] "RemoveContainer" containerID="73be814c62ef661f95e47e455ce0366e6aac65f49910616aa57caac0d9b60fed" Sep 10 04:04:38.937261 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:04:38.937031 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7zgvh_llm(4dc064fe-8790-42ff-8c51-a4dce6cd3731)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" podUID="4dc064fe-8790-42ff-8c51-a4dce6cd3731" Sep 10 04:04:46.939164 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:04:46.939132 2581 scope.go:117] "RemoveContainer" containerID="3cf054fbd398ad0be58323eb1c8032af151525e64ac1f1bc9d3e7ea2f3335604" Sep 10 04:04:46.939742 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:04:46.939346 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7lp54_llm(2c6ade3f-3c19-4b01-999f-f568494a937f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" podUID="2c6ade3f-3c19-4b01-999f-f568494a937f" Sep 10 04:04:49.936949 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:04:49.936913 2581 scope.go:117] "RemoveContainer" containerID="73be814c62ef661f95e47e455ce0366e6aac65f49910616aa57caac0d9b60fed" Sep 10 04:04:50.717045 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:04:50.717011 2581 generic.go:356] "Generic (PLEG): container finished" podID="4dc064fe-8790-42ff-8c51-a4dce6cd3731" containerID="4b74121466a3fe6cbbc80d0a9c400e3d7f3c049a975ac6a49fc33014daa26222" exitCode=0 Sep 10 04:04:50.717204 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:04:50.717072 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" event={"ID":"4dc064fe-8790-42ff-8c51-a4dce6cd3731","Type":"ContainerDied","Data":"4b74121466a3fe6cbbc80d0a9c400e3d7f3c049a975ac6a49fc33014daa26222"} Sep 10 04:04:50.717204 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:04:50.717140 2581 scope.go:117] "RemoveContainer" containerID="73be814c62ef661f95e47e455ce0366e6aac65f49910616aa57caac0d9b60fed" Sep 10 04:04:50.717482 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:04:50.717463 2581 scope.go:117] "RemoveContainer" containerID="4b74121466a3fe6cbbc80d0a9c400e3d7f3c049a975ac6a49fc33014daa26222" Sep 10 04:04:50.717687 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:04:50.717666 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7zgvh_llm(4dc064fe-8790-42ff-8c51-a4dce6cd3731)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" podUID="4dc064fe-8790-42ff-8c51-a4dce6cd3731" Sep 10 04:04:58.052464 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:04:58.052413 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" Sep 10 04:04:58.052918 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:04:58.052491 2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" Sep 10 04:04:58.052918 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:04:58.052514 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" Sep 10 04:04:58.052918 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:04:58.052528 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" Sep 10 04:04:58.052918 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:04:58.052856 2581 scope.go:117] "RemoveContainer" containerID="4b74121466a3fe6cbbc80d0a9c400e3d7f3c049a975ac6a49fc33014daa26222" Sep 10 04:04:58.053113 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:04:58.053056 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7zgvh_llm(4dc064fe-8790-42ff-8c51-a4dce6cd3731)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" podUID="4dc064fe-8790-42ff-8c51-a4dce6cd3731" Sep 10 04:04:59.937369 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:04:59.937326 2581 scope.go:117] "RemoveContainer" containerID="3cf054fbd398ad0be58323eb1c8032af151525e64ac1f1bc9d3e7ea2f3335604" Sep 10 04:04:59.937800 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:04:59.937521 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7lp54_llm(2c6ade3f-3c19-4b01-999f-f568494a937f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" podUID="2c6ade3f-3c19-4b01-999f-f568494a937f" Sep 10 04:05:09.937744 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:05:09.937693 2581 scope.go:117] "RemoveContainer" containerID="4b74121466a3fe6cbbc80d0a9c400e3d7f3c049a975ac6a49fc33014daa26222" Sep 10 04:05:09.938268 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:05:09.937885 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7zgvh_llm(4dc064fe-8790-42ff-8c51-a4dce6cd3731)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" podUID="4dc064fe-8790-42ff-8c51-a4dce6cd3731" Sep 10 04:05:11.937231 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:05:11.937200 2581 scope.go:117] "RemoveContainer" containerID="3cf054fbd398ad0be58323eb1c8032af151525e64ac1f1bc9d3e7ea2f3335604" Sep 10 04:05:11.937595 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:05:11.937366 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7lp54_llm(2c6ade3f-3c19-4b01-999f-f568494a937f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" podUID="2c6ade3f-3c19-4b01-999f-f568494a937f" Sep 10 04:05:22.937227 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:05:22.937189 2581 scope.go:117] "RemoveContainer" containerID="3cf054fbd398ad0be58323eb1c8032af151525e64ac1f1bc9d3e7ea2f3335604" Sep 10 04:05:22.937813 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:05:22.937403 2581 scope.go:117] "RemoveContainer" containerID="4b74121466a3fe6cbbc80d0a9c400e3d7f3c049a975ac6a49fc33014daa26222" Sep 10 04:05:22.937813 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:05:22.937412 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7lp54_llm(2c6ade3f-3c19-4b01-999f-f568494a937f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" podUID="2c6ade3f-3c19-4b01-999f-f568494a937f" Sep 10 04:05:22.937813 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:05:22.937627 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7zgvh_llm(4dc064fe-8790-42ff-8c51-a4dce6cd3731)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" podUID="4dc064fe-8790-42ff-8c51-a4dce6cd3731" Sep 10 04:05:33.937201 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:05:33.937167 2581 scope.go:117] "RemoveContainer" containerID="3cf054fbd398ad0be58323eb1c8032af151525e64ac1f1bc9d3e7ea2f3335604" Sep 10 04:05:33.937641 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:05:33.937340 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7lp54_llm(2c6ade3f-3c19-4b01-999f-f568494a937f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" podUID="2c6ade3f-3c19-4b01-999f-f568494a937f" Sep 10 04:05:37.937198 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:05:37.937169 2581 scope.go:117] "RemoveContainer" containerID="4b74121466a3fe6cbbc80d0a9c400e3d7f3c049a975ac6a49fc33014daa26222" Sep 10 04:05:37.937584 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:05:37.937344 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7zgvh_llm(4dc064fe-8790-42ff-8c51-a4dce6cd3731)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" podUID="4dc064fe-8790-42ff-8c51-a4dce6cd3731" Sep 10 04:05:45.937410 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:05:45.937372 2581 scope.go:117] "RemoveContainer" containerID="3cf054fbd398ad0be58323eb1c8032af151525e64ac1f1bc9d3e7ea2f3335604" Sep 10 04:05:45.937819 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:05:45.937605 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7lp54_llm(2c6ade3f-3c19-4b01-999f-f568494a937f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" podUID="2c6ade3f-3c19-4b01-999f-f568494a937f" Sep 10 04:05:52.937066 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:05:52.937029 2581 scope.go:117] "RemoveContainer" containerID="4b74121466a3fe6cbbc80d0a9c400e3d7f3c049a975ac6a49fc33014daa26222" Sep 10 04:05:52.937412 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:05:52.937200 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7zgvh_llm(4dc064fe-8790-42ff-8c51-a4dce6cd3731)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" podUID="4dc064fe-8790-42ff-8c51-a4dce6cd3731" Sep 10 04:05:57.937543 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:05:57.937512 2581 scope.go:117] "RemoveContainer" containerID="3cf054fbd398ad0be58323eb1c8032af151525e64ac1f1bc9d3e7ea2f3335604" Sep 10 04:05:57.937962 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:05:57.937696 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7lp54_llm(2c6ade3f-3c19-4b01-999f-f568494a937f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" podUID="2c6ade3f-3c19-4b01-999f-f568494a937f" Sep 10 04:06:04.937158 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:04.937125 2581 scope.go:117] "RemoveContainer" containerID="4b74121466a3fe6cbbc80d0a9c400e3d7f3c049a975ac6a49fc33014daa26222" Sep 10 04:06:04.939655 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:06:04.937310 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7zgvh_llm(4dc064fe-8790-42ff-8c51-a4dce6cd3731)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" podUID="4dc064fe-8790-42ff-8c51-a4dce6cd3731" Sep 10 04:06:09.428303 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:09.428275 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-psjjd_e2c55df8-569f-4e2f-a317-ad282f00bddc/server/0.log" Sep 10 04:06:09.540872 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:09.540837 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-lzd2s_74e98872-0bca-4109-9865-2611f1493cde/manager/0.log" Sep 10 04:06:09.783549 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:09.783528 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-797557847b-5kp5j_8a0e0403-cfc8-485f-acf8-c56389c13825/copy-manifests/0.log" Sep 10 04:06:09.789584 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:09.789557 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-797557847b-5kp5j_8a0e0403-cfc8-485f-acf8-c56389c13825/manager/0.log" Sep 10 04:06:09.937406 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:09.937372 2581 scope.go:117] "RemoveContainer" containerID="3cf054fbd398ad0be58323eb1c8032af151525e64ac1f1bc9d3e7ea2f3335604" Sep 10 04:06:09.937561 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:06:09.937530 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7lp54_llm(2c6ade3f-3c19-4b01-999f-f568494a937f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" podUID="2c6ade3f-3c19-4b01-999f-f568494a937f" Sep 10 04:06:12.196282 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:12.196254 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799chwtkx_db5782e3-ade9-48e9-8e2c-2aa4e36c4c15/istio-proxy/0.log" Sep 10 04:06:12.307804 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:12.307777 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-8tmrn_c13ee959-189a-4d0a-adec-2f14afab064a/discovery/0.log" Sep 10 04:06:12.639917 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:12.639882 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-j59lv_00a43ad9-fe36-45f2-bdab-8b844574373d/istio-proxy/1.log" Sep 10 04:06:13.333797 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:13.333770 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-859856b874-7lp54_2c6ade3f-3c19-4b01-999f-f568494a937f/main/5.log" Sep 10 04:06:13.454346 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:13.454320 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-6685d86975-7zgvh_4dc064fe-8790-42ff-8c51-a4dce6cd3731/main/5.log" Sep 10 04:06:17.936794 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:17.936761 2581 scope.go:117] "RemoveContainer" containerID="4b74121466a3fe6cbbc80d0a9c400e3d7f3c049a975ac6a49fc33014daa26222" Sep 10 04:06:17.937208 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:06:17.936935 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7zgvh_llm(4dc064fe-8790-42ff-8c51-a4dce6cd3731)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" podUID="4dc064fe-8790-42ff-8c51-a4dce6cd3731" Sep 10 04:06:22.532931 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:22.532898 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-2z94t_72f457e8-c3eb-4b62-81e6-10cdd0f78d63/global-pull-secret-syncer/0.log" Sep 10 04:06:22.722466 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:22.722434 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-sjkng_271dd21f-0eed-4858-810a-687acbea64ec/konnectivity-agent/0.log" Sep 10 04:06:22.789947 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:22.789871 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-77.ec2.internal_81cf177ab4229900eb060a4c89e0a554/haproxy/0.log" Sep 10 04:06:22.937241 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:22.937206 2581 scope.go:117] "RemoveContainer" containerID="3cf054fbd398ad0be58323eb1c8032af151525e64ac1f1bc9d3e7ea2f3335604" Sep 10 04:06:22.937437 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:06:22.937406 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7lp54_llm(2c6ade3f-3c19-4b01-999f-f568494a937f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" podUID="2c6ade3f-3c19-4b01-999f-f568494a937f" Sep 10 04:06:29.286582 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:29.286551 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-jc5tg_7a060837-9c83-47f5-8425-df493263a67d/monitoring-plugin/0.log" Sep 10 04:06:29.317142 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:29.317114 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dsxrz_df73b602-cb9a-4072-bd26-88bef578eef5/node-exporter/0.log" Sep 10 04:06:29.342112 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:29.342087 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dsxrz_df73b602-cb9a-4072-bd26-88bef578eef5/kube-rbac-proxy/0.log" Sep 10 04:06:29.366992 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:29.366954 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dsxrz_df73b602-cb9a-4072-bd26-88bef578eef5/init-textfile/0.log" Sep 10 04:06:29.890102 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:29.890073 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-rcplh_853ceb7f-4c96-4f16-af84-d6f89525afcb/prometheus-operator-admission-webhook/0.log" Sep 10 04:06:31.937384 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:31.937360 2581 scope.go:117] "RemoveContainer" containerID="4b74121466a3fe6cbbc80d0a9c400e3d7f3c049a975ac6a49fc33014daa26222" Sep 10 04:06:31.937751 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:06:31.937527 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7zgvh_llm(4dc064fe-8790-42ff-8c51-a4dce6cd3731)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" podUID="4dc064fe-8790-42ff-8c51-a4dce6cd3731" Sep 10 04:06:32.655102 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:32.655067 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5qpb6/perf-node-gather-daemonset-5cfbt"] Sep 10 04:06:32.658452 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:32.658435 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-5cfbt" Sep 10 04:06:32.661015 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:32.660947 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5qpb6\"/\"openshift-service-ca.crt\"" Sep 10 04:06:32.661116 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:32.661098 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5qpb6\"/\"default-dockercfg-fppwb\"" Sep 10 04:06:32.662229 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:32.662213 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5qpb6\"/\"kube-root-ca.crt\"" Sep 10 04:06:32.702288 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:32.702257 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5qpb6/perf-node-gather-daemonset-5cfbt"] Sep 10 04:06:32.755348 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:32.755306 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/98b3621f-403d-4539-aa55-e62f142c8fc6-sys\") pod \"perf-node-gather-daemonset-5cfbt\" (UID: \"98b3621f-403d-4539-aa55-e62f142c8fc6\") " pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-5cfbt" Sep 10 04:06:32.755514 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:32.755353 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/98b3621f-403d-4539-aa55-e62f142c8fc6-proc\") pod \"perf-node-gather-daemonset-5cfbt\" (UID: \"98b3621f-403d-4539-aa55-e62f142c8fc6\") " pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-5cfbt" Sep 10 04:06:32.755514 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:32.755382 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/98b3621f-403d-4539-aa55-e62f142c8fc6-lib-modules\") pod \"perf-node-gather-daemonset-5cfbt\" (UID: \"98b3621f-403d-4539-aa55-e62f142c8fc6\") " pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-5cfbt" Sep 10 04:06:32.755514 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:32.755418 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/98b3621f-403d-4539-aa55-e62f142c8fc6-podres\") pod \"perf-node-gather-daemonset-5cfbt\" (UID: \"98b3621f-403d-4539-aa55-e62f142c8fc6\") " pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-5cfbt" Sep 10 04:06:32.755514 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:32.755437 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7sq4\" (UniqueName: \"kubernetes.io/projected/98b3621f-403d-4539-aa55-e62f142c8fc6-kube-api-access-t7sq4\") pod \"perf-node-gather-daemonset-5cfbt\" (UID: \"98b3621f-403d-4539-aa55-e62f142c8fc6\") " pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-5cfbt" Sep 10 04:06:32.856738 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:32.856693 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/98b3621f-403d-4539-aa55-e62f142c8fc6-podres\") pod \"perf-node-gather-daemonset-5cfbt\" (UID: \"98b3621f-403d-4539-aa55-e62f142c8fc6\") " pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-5cfbt" Sep 10 04:06:32.856930 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:32.856758 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7sq4\" (UniqueName: \"kubernetes.io/projected/98b3621f-403d-4539-aa55-e62f142c8fc6-kube-api-access-t7sq4\") pod \"perf-node-gather-daemonset-5cfbt\" (UID: \"98b3621f-403d-4539-aa55-e62f142c8fc6\") " pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-5cfbt" Sep 10 04:06:32.856930 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:32.856823 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/98b3621f-403d-4539-aa55-e62f142c8fc6-sys\") pod \"perf-node-gather-daemonset-5cfbt\" (UID: \"98b3621f-403d-4539-aa55-e62f142c8fc6\") " pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-5cfbt" Sep 10 04:06:32.856930 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:32.856867 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/98b3621f-403d-4539-aa55-e62f142c8fc6-podres\") pod \"perf-node-gather-daemonset-5cfbt\" (UID: \"98b3621f-403d-4539-aa55-e62f142c8fc6\") " pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-5cfbt" Sep 10 04:06:32.856930 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:32.856879 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/98b3621f-403d-4539-aa55-e62f142c8fc6-proc\") pod \"perf-node-gather-daemonset-5cfbt\" (UID: \"98b3621f-403d-4539-aa55-e62f142c8fc6\") " pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-5cfbt" Sep 10 04:06:32.856930 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:32.856916 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/98b3621f-403d-4539-aa55-e62f142c8fc6-lib-modules\") pod \"perf-node-gather-daemonset-5cfbt\" (UID: \"98b3621f-403d-4539-aa55-e62f142c8fc6\") " pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-5cfbt" Sep 10 04:06:32.856930 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:32.856930 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/98b3621f-403d-4539-aa55-e62f142c8fc6-proc\") pod \"perf-node-gather-daemonset-5cfbt\" (UID: \"98b3621f-403d-4539-aa55-e62f142c8fc6\") " pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-5cfbt" Sep 10 04:06:32.857180 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:32.856962 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/98b3621f-403d-4539-aa55-e62f142c8fc6-sys\") pod \"perf-node-gather-daemonset-5cfbt\" (UID: \"98b3621f-403d-4539-aa55-e62f142c8fc6\") " pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-5cfbt" Sep 10 04:06:32.857180 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:32.857096 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/98b3621f-403d-4539-aa55-e62f142c8fc6-lib-modules\") pod \"perf-node-gather-daemonset-5cfbt\" (UID: \"98b3621f-403d-4539-aa55-e62f142c8fc6\") " pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-5cfbt" Sep 10 04:06:32.865317 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:32.865293 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7sq4\" (UniqueName: \"kubernetes.io/projected/98b3621f-403d-4539-aa55-e62f142c8fc6-kube-api-access-t7sq4\") pod \"perf-node-gather-daemonset-5cfbt\" (UID: \"98b3621f-403d-4539-aa55-e62f142c8fc6\") " pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-5cfbt" Sep 10 04:06:32.907121 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:32.907043 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7f44969586-qn22r_ed4eb216-8475-492d-905f-acbf98058a5f/console/0.log" Sep 10 04:06:32.967796 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:32.967757 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-5cfbt" Sep 10 04:06:33.240319 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:33.240297 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5qpb6/perf-node-gather-daemonset-5cfbt"] Sep 10 04:06:33.242591 ip-10-0-142-77 kubenswrapper[2581]: W0910 04:06:33.242551 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod98b3621f_403d_4539_aa55_e62f142c8fc6.slice/crio-a59ec3c719b96d331f220d06ecb7d818608dd2f7bc7dcdac201060182d96c88e WatchSource:0}: Error finding container a59ec3c719b96d331f220d06ecb7d818608dd2f7bc7dcdac201060182d96c88e: Status 404 returned error can't find the container with id a59ec3c719b96d331f220d06ecb7d818608dd2f7bc7dcdac201060182d96c88e Sep 10 04:06:34.142891 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:34.142862 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-5cfbt" event={"ID":"98b3621f-403d-4539-aa55-e62f142c8fc6","Type":"ContainerStarted","Data":"134d2de87c18a0f4433b59c0ee33ec04eb2b312ba388828185e5cbbcefed3632"} Sep 10 04:06:34.143276 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:34.142908 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-5cfbt" event={"ID":"98b3621f-403d-4539-aa55-e62f142c8fc6","Type":"ContainerStarted","Data":"a59ec3c719b96d331f220d06ecb7d818608dd2f7bc7dcdac201060182d96c88e"} Sep 10 04:06:34.143276 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:34.142960 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-5cfbt" Sep 10 04:06:34.158564 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:34.158521 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-5cfbt" podStartSLOduration=2.158507383 podStartE2EDuration="2.158507383s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 04:06:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 04:06:34.157053124 +0000 UTC m=+1107.682432230" watchObservedRunningTime="2026-09-10 04:06:34.158507383 +0000 UTC m=+1107.683886489" Sep 10 04:06:34.753112 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:34.753085 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-kzds7_184e14a2-418b-41da-8635-23ea4c965dbc/dns/0.log" Sep 10 04:06:34.773105 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:34.773081 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-kzds7_184e14a2-418b-41da-8635-23ea4c965dbc/kube-rbac-proxy/0.log" Sep 10 04:06:34.820802 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:34.820776 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7lpdd_29111c2f-bef9-4076-9014-7f041915d6df/dns-node-resolver/0.log" Sep 10 04:06:34.937148 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:34.937123 2581 scope.go:117] "RemoveContainer" containerID="3cf054fbd398ad0be58323eb1c8032af151525e64ac1f1bc9d3e7ea2f3335604" Sep 10 04:06:34.937312 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:06:34.937276 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7lp54_llm(2c6ade3f-3c19-4b01-999f-f568494a937f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" podUID="2c6ade3f-3c19-4b01-999f-f568494a937f" Sep 10 04:06:35.422800 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:35.422772 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-r58nd_b4c0f4fb-e2d6-4ed5-969e-ec1ff4aaff92/node-ca/0.log" Sep 10 04:06:36.344206 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:36.344179 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799chwtkx_db5782e3-ade9-48e9-8e2c-2aa4e36c4c15/istio-proxy/0.log" Sep 10 04:06:36.386542 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:36.386513 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-8tmrn_c13ee959-189a-4d0a-adec-2f14afab064a/discovery/0.log" Sep 10 04:06:36.454278 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:36.454246 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-j59lv_00a43ad9-fe36-45f2-bdab-8b844574373d/istio-proxy/1.log" Sep 10 04:06:36.454741 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:36.454519 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-j59lv_00a43ad9-fe36-45f2-bdab-8b844574373d/istio-proxy/0.log" Sep 10 04:06:37.174956 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:37.174927 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-cmdjp_59ba01f7-c3bf-4ccb-8a19-f69bbda08d5f/serve-healthcheck-canary/0.log" Sep 10 04:06:37.789448 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:37.789422 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hj47d_53b35768-5fa3-41d1-87f2-4fb4f7adc140/kube-rbac-proxy/0.log" Sep 10 04:06:37.811793 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:37.811771 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hj47d_53b35768-5fa3-41d1-87f2-4fb4f7adc140/exporter/0.log" Sep 10 04:06:37.831952 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:37.831931 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hj47d_53b35768-5fa3-41d1-87f2-4fb4f7adc140/extractor/0.log" Sep 10 04:06:40.092382 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:40.092354 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-psjjd_e2c55df8-569f-4e2f-a317-ad282f00bddc/server/0.log" Sep 10 04:06:40.138141 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:40.138116 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-lzd2s_74e98872-0bca-4109-9865-2611f1493cde/manager/0.log" Sep 10 04:06:40.156918 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:40.156897 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5qpb6/perf-node-gather-daemonset-5cfbt" Sep 10 04:06:40.258400 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:40.258370 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-797557847b-5kp5j_8a0e0403-cfc8-485f-acf8-c56389c13825/manager/0.log" Sep 10 04:06:40.277880 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:40.277859 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-797557847b-5kp5j_8a0e0403-cfc8-485f-acf8-c56389c13825/copy-manifests/0.log" Sep 10 04:06:41.588119 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:41.588089 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-78c97bdccc-7zmx8_15ab54e8-0ae3-49e2-8eaf-ecc8ca8bf39b/manager/0.log" Sep 10 04:06:43.305073 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:43.305049 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799chwtkx_db5782e3-ade9-48e9-8e2c-2aa4e36c4c15/istio-proxy/0.log" Sep 10 04:06:43.340696 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:43.340670 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-8tmrn_c13ee959-189a-4d0a-adec-2f14afab064a/discovery/0.log" Sep 10 04:06:43.390706 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:43.390678 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-j59lv_00a43ad9-fe36-45f2-bdab-8b844574373d/istio-proxy/1.log" Sep 10 04:06:43.391095 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:43.391083 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-j59lv_00a43ad9-fe36-45f2-bdab-8b844574373d/istio-proxy/0.log" Sep 10 04:06:44.563376 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:44.563347 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799chwtkx_db5782e3-ade9-48e9-8e2c-2aa4e36c4c15/istio-proxy/0.log" Sep 10 04:06:44.596899 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:44.596874 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-8tmrn_c13ee959-189a-4d0a-adec-2f14afab064a/discovery/0.log" Sep 10 04:06:44.641152 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:44.641125 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-j59lv_00a43ad9-fe36-45f2-bdab-8b844574373d/istio-proxy/1.log" Sep 10 04:06:44.641682 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:44.641664 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-j59lv_00a43ad9-fe36-45f2-bdab-8b844574373d/istio-proxy/0.log" Sep 10 04:06:46.943648 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:46.940939 2581 scope.go:117] "RemoveContainer" containerID="3cf054fbd398ad0be58323eb1c8032af151525e64ac1f1bc9d3e7ea2f3335604" Sep 10 04:06:46.943648 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:06:46.941278 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-7lp54_llm(2c6ade3f-3c19-4b01-999f-f568494a937f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-7lp54" podUID="2c6ade3f-3c19-4b01-999f-f568494a937f" Sep 10 04:06:46.943648 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:46.941916 2581 scope.go:117] "RemoveContainer" containerID="4b74121466a3fe6cbbc80d0a9c400e3d7f3c049a975ac6a49fc33014daa26222" Sep 10 04:06:46.943648 ip-10-0-142-77 kubenswrapper[2581]: E0910 04:06:46.942177 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7zgvh_llm(4dc064fe-8790-42ff-8c51-a4dce6cd3731)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7zgvh" podUID="4dc064fe-8790-42ff-8c51-a4dce6cd3731" Sep 10 04:06:48.181176 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:48.181152 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6tbmr_24e969cd-fc73-4628-bcd8-0abb930691d8/kube-multus-additional-cni-plugins/0.log" Sep 10 04:06:48.200902 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:48.200875 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6tbmr_24e969cd-fc73-4628-bcd8-0abb930691d8/egress-router-binary-copy/0.log" Sep 10 04:06:48.219797 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:48.219775 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6tbmr_24e969cd-fc73-4628-bcd8-0abb930691d8/cni-plugins/0.log" Sep 10 04:06:48.238207 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:48.238182 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6tbmr_24e969cd-fc73-4628-bcd8-0abb930691d8/bond-cni-plugin/0.log" Sep 10 04:06:48.259273 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:48.259248 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6tbmr_24e969cd-fc73-4628-bcd8-0abb930691d8/routeoverride-cni/0.log" Sep 10 04:06:48.278589 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:48.278566 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6tbmr_24e969cd-fc73-4628-bcd8-0abb930691d8/whereabouts-cni-bincopy/0.log" Sep 10 04:06:48.297179 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:48.297154 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6tbmr_24e969cd-fc73-4628-bcd8-0abb930691d8/whereabouts-cni/0.log" Sep 10 04:06:48.690747 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:48.690723 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-wnlbc_1d9fdaf3-79da-4413-8a44-346fe29074be/kube-multus/0.log" Sep 10 04:06:48.769323 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:48.769296 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-htpt9_fca9ba6d-4934-47e4-8be5-4296c7a6c530/network-metrics-daemon/0.log" Sep 10 04:06:48.796295 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:48.796269 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-htpt9_fca9ba6d-4934-47e4-8be5-4296c7a6c530/kube-rbac-proxy/0.log" Sep 10 04:06:49.741570 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:49.741535 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cqmkb_f0b85b12-5116-477e-86d0-2946bc920ded/ovn-controller/0.log" Sep 10 04:06:49.761493 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:49.761468 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cqmkb_f0b85b12-5116-477e-86d0-2946bc920ded/ovn-acl-logging/0.log" Sep 10 04:06:49.786897 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:49.786875 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cqmkb_f0b85b12-5116-477e-86d0-2946bc920ded/kube-rbac-proxy-node/0.log" Sep 10 04:06:49.805786 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:49.805763 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cqmkb_f0b85b12-5116-477e-86d0-2946bc920ded/kube-rbac-proxy-ovn-metrics/0.log" Sep 10 04:06:49.822718 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:49.822699 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cqmkb_f0b85b12-5116-477e-86d0-2946bc920ded/northd/0.log" Sep 10 04:06:49.843105 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:49.843085 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cqmkb_f0b85b12-5116-477e-86d0-2946bc920ded/nbdb/0.log" Sep 10 04:06:49.864038 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:49.864019 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cqmkb_f0b85b12-5116-477e-86d0-2946bc920ded/sbdb/0.log" Sep 10 04:06:49.902259 ip-10-0-142-77 kubenswrapper[2581]: I0910 04:06:49.902234 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cqmkb_f0b85b12-5116-477e-86d0-2946bc920ded/ovnkube-controller/0.log"