Oct 30 22:05:05.996816 ip-10-0-142-24 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Oct 30 22:05:05.996830 ip-10-0-142-24 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Oct 30 22:05:05.996838 ip-10-0-142-24 systemd[1]: kubelet.service: Failed with result 'resources'. Oct 30 22:05:05.997175 ip-10-0-142-24 systemd[1]: Failed to start Kubernetes Kubelet. Oct 30 22:05:16.104509 ip-10-0-142-24 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Oct 30 22:05:16.104531 ip-10-0-142-24 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot f366a10e7b184883b9c8d788d9a7e2ec -- Oct 30 22:07:20.944497 ip-10-0-142-24 systemd[1]: Starting Kubernetes Kubelet... Oct 30 22:07:21.346257 ip-10-0-142-24 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 22:07:21.346257 ip-10-0-142-24 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Oct 30 22:07:21.346257 ip-10-0-142-24 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 22:07:21.346257 ip-10-0-142-24 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 22:07:21.350164 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350083 2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 22:07:21.350164 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350164 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 22:07:21.350247 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350168 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 22:07:21.350247 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350171 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 22:07:21.350247 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350175 2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 22:07:21.350247 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350178 2581 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 22:07:21.350247 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350180 2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 22:07:21.350247 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350183 2581 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 22:07:21.350247 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350186 2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 22:07:21.350247 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350188 2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 22:07:21.350247 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350191 2581 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 22:07:21.350247 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350194 2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 22:07:21.350247 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350197 2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 22:07:21.350247 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350200 2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 22:07:21.350247 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350203 2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 22:07:21.350247 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350205 2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 22:07:21.350247 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350208 2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 22:07:21.350247 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350211 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 22:07:21.350247 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350213 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 22:07:21.350247 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350228 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 22:07:21.350247 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350231 2581 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 22:07:21.350247 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350234 2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 22:07:21.350710 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350236 2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 22:07:21.350710 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350239 2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 22:07:21.350710 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350242 2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 22:07:21.350710 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350245 2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 22:07:21.350710 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350248 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 22:07:21.350710 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350251 2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 22:07:21.350710 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350254 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 22:07:21.350710 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350258 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 22:07:21.350710 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350261 2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 22:07:21.350710 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350264 2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 22:07:21.350710 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350266 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 22:07:21.350710 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350269 2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 22:07:21.350710 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350272 2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 22:07:21.350710 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350274 2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 22:07:21.350710 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350277 2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 22:07:21.350710 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350279 2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 22:07:21.350710 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350282 2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 22:07:21.350710 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350284 2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 22:07:21.350710 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350287 2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 22:07:21.351152 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350290 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 22:07:21.351152 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350293 2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 22:07:21.351152 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350296 2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 22:07:21.351152 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350298 2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 22:07:21.351152 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350301 2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 22:07:21.351152 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350303 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 22:07:21.351152 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350306 2581 feature_gate.go:386] unrecognized feature gate: Example Oct 30 22:07:21.351152 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350309 2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 22:07:21.351152 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350311 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 22:07:21.351152 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350314 2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 22:07:21.351152 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350316 2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 22:07:21.351152 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350319 2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 22:07:21.351152 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350321 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 22:07:21.351152 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350324 2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 22:07:21.351152 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350327 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 22:07:21.351152 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350330 2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 22:07:21.351152 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350332 2581 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 22:07:21.351152 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350335 2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 22:07:21.351152 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350338 2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 22:07:21.351152 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350340 2581 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 22:07:21.351634 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350343 2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 22:07:21.351634 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350346 2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 22:07:21.351634 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350349 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 22:07:21.351634 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350351 2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 22:07:21.351634 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350354 2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 22:07:21.351634 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350356 2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 22:07:21.351634 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350361 2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 22:07:21.351634 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350365 2581 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 22:07:21.351634 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350369 2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 22:07:21.351634 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350371 2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 22:07:21.351634 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350374 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 22:07:21.351634 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350378 2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 22:07:21.351634 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350380 2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 22:07:21.351634 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350383 2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 22:07:21.351634 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350387 2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 22:07:21.351634 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350391 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 22:07:21.351634 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350394 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 22:07:21.351634 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350397 2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 22:07:21.351634 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350400 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 22:07:21.352080 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350403 2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 22:07:21.352080 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350405 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 22:07:21.352080 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350408 2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 22:07:21.352080 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350411 2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 22:07:21.352080 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350413 2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 22:07:21.352080 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350416 2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 22:07:21.352080 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350418 2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 22:07:21.352080 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350421 2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 22:07:21.352080 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350424 2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 22:07:21.352080 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350427 2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 22:07:21.352080 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350430 2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 22:07:21.352080 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350432 2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 22:07:21.352080 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350435 2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 22:07:21.352080 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350438 2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 22:07:21.352080 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350441 2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 22:07:21.352080 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350444 2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 22:07:21.352080 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350447 2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 22:07:21.352080 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350449 2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 22:07:21.352080 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350452 2581 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 22:07:21.352080 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350455 2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 22:07:21.352600 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350457 2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 22:07:21.352600 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350460 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 22:07:21.352600 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350462 2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 22:07:21.352600 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350465 2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 22:07:21.352600 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350467 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 22:07:21.352600 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.350470 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 22:07:21.352600 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351261 2581 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 22:07:21.352600 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351267 2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 22:07:21.352600 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351270 2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 22:07:21.352600 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351273 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 22:07:21.352600 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351276 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 22:07:21.352600 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351279 2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 22:07:21.352600 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351282 2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 22:07:21.352600 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351285 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 22:07:21.352600 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351288 2581 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 22:07:21.352600 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351290 2581 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 22:07:21.352600 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351293 2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 22:07:21.352600 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351297 2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 22:07:21.352600 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351299 2581 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 22:07:21.352600 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351302 2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 22:07:21.353172 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351307 2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 22:07:21.353172 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351311 2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 22:07:21.353172 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351314 2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 22:07:21.353172 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351317 2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 22:07:21.353172 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351320 2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 22:07:21.353172 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351323 2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 22:07:21.353172 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351326 2581 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 22:07:21.353172 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351329 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 22:07:21.353172 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351332 2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 22:07:21.353172 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351334 2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 22:07:21.353172 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351337 2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 22:07:21.353172 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351340 2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 22:07:21.353172 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351342 2581 feature_gate.go:386] unrecognized feature gate: Example Oct 30 22:07:21.353172 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351345 2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 22:07:21.353172 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351347 2581 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 22:07:21.353172 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351350 2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 22:07:21.353172 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351352 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 22:07:21.353172 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351355 2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 22:07:21.353172 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351358 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 22:07:21.353633 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351361 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 22:07:21.353633 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351363 2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 22:07:21.353633 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351366 2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 22:07:21.353633 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351368 2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 22:07:21.353633 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351372 2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 22:07:21.353633 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351376 2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 22:07:21.353633 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351380 2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 22:07:21.353633 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351383 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 22:07:21.353633 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351386 2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 22:07:21.353633 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351389 2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 22:07:21.353633 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351392 2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 22:07:21.353633 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351395 2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 22:07:21.353633 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351397 2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 22:07:21.353633 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351400 2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 22:07:21.353633 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351403 2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 22:07:21.353633 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351405 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 22:07:21.353633 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351408 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 22:07:21.353633 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351410 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 22:07:21.353633 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351413 2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 22:07:21.354087 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351415 2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 22:07:21.354087 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351418 2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 22:07:21.354087 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351421 2581 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 22:07:21.354087 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351424 2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 22:07:21.354087 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351427 2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 22:07:21.354087 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351429 2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 22:07:21.354087 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351432 2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 22:07:21.354087 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351435 2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 22:07:21.354087 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351438 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 22:07:21.354087 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351440 2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 22:07:21.354087 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351443 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 22:07:21.354087 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351445 2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 22:07:21.354087 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351448 2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 22:07:21.354087 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351452 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 22:07:21.354087 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351454 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 22:07:21.354087 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351457 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 22:07:21.354087 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351459 2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 22:07:21.354087 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351462 2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 22:07:21.354087 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351465 2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 22:07:21.354087 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351468 2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 22:07:21.354577 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351470 2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 22:07:21.354577 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351473 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 22:07:21.354577 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351475 2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 22:07:21.354577 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351478 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 22:07:21.354577 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351481 2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 22:07:21.354577 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351483 2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 22:07:21.354577 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351485 2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 22:07:21.354577 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351488 2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 22:07:21.354577 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351490 2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 22:07:21.354577 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351493 2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 22:07:21.354577 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351495 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 22:07:21.354577 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351498 2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 22:07:21.354577 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351501 2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 22:07:21.354577 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351504 2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 22:07:21.354577 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351506 2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 22:07:21.354577 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351509 2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 22:07:21.354577 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351512 2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 22:07:21.354577 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351515 2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 22:07:21.354577 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351517 2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 22:07:21.354577 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351520 2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 22:07:21.355045 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351522 2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 22:07:21.355045 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351525 2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 22:07:21.355045 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351527 2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 22:07:21.355045 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351530 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 22:07:21.355045 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351532 2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 22:07:21.355045 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351535 2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 22:07:21.355045 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351537 2581 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 22:07:21.355045 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351540 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 22:07:21.355045 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351542 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 22:07:21.355045 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351547 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 22:07:21.355045 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351549 2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 22:07:21.355045 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351552 2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 22:07:21.355045 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351554 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 22:07:21.355045 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.351557 2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 22:07:21.355045 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352089 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 22:07:21.355045 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352095 2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 22:07:21.355045 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352116 2581 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 22:07:21.355045 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352120 2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 22:07:21.355045 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352123 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 22:07:21.355045 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352126 2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 22:07:21.355529 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352129 2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 22:07:21.355529 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352131 2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 22:07:21.355529 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352134 2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 22:07:21.355529 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352137 2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 22:07:21.355529 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352139 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 22:07:21.355529 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352142 2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 22:07:21.355529 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352144 2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 22:07:21.355529 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352153 2581 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 22:07:21.355529 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352155 2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 22:07:21.355529 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352158 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 22:07:21.355529 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352161 2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 22:07:21.355529 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352163 2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 22:07:21.355529 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352166 2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 22:07:21.355529 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352168 2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 22:07:21.355529 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352171 2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 22:07:21.355529 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352174 2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 22:07:21.355529 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352176 2581 feature_gate.go:386] unrecognized feature gate: Example Oct 30 22:07:21.355529 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352179 2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 22:07:21.355529 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352181 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 22:07:21.355529 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352184 2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 22:07:21.355996 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352189 2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 22:07:21.355996 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352192 2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 22:07:21.355996 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352195 2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 22:07:21.355996 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352197 2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 22:07:21.355996 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352200 2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 22:07:21.355996 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352203 2581 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 22:07:21.355996 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352206 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 22:07:21.355996 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352208 2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 22:07:21.355996 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352211 2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 22:07:21.355996 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352213 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 22:07:21.355996 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352230 2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 22:07:21.355996 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352232 2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 22:07:21.355996 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352235 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 22:07:21.355996 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352238 2581 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 22:07:21.355996 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352240 2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 22:07:21.355996 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352243 2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 22:07:21.355996 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352246 2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 22:07:21.355996 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352248 2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 22:07:21.355996 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352251 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 22:07:21.355996 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352253 2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 22:07:21.356470 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352256 2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 22:07:21.356470 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352259 2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 22:07:21.356470 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352261 2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 22:07:21.356470 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352264 2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 22:07:21.356470 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352266 2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 22:07:21.356470 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352269 2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 22:07:21.356470 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352271 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 22:07:21.356470 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352274 2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 22:07:21.356470 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352277 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 22:07:21.356470 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352280 2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 22:07:21.356470 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352282 2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 22:07:21.356470 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352285 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 22:07:21.356470 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352289 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 22:07:21.356470 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352292 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 22:07:21.356470 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352295 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 22:07:21.356470 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352297 2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 22:07:21.356470 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352300 2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 22:07:21.356470 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352302 2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 22:07:21.356470 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352305 2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 22:07:21.356911 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352308 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 22:07:21.356911 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352310 2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 22:07:21.356911 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352313 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 22:07:21.356911 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352315 2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 22:07:21.356911 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352318 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 22:07:21.356911 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352320 2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 22:07:21.356911 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352323 2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 22:07:21.356911 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352326 2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 22:07:21.356911 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352328 2581 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 22:07:21.356911 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352331 2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 22:07:21.356911 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352335 2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 22:07:21.356911 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352339 2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 22:07:21.356911 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352342 2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 22:07:21.356911 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352345 2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 22:07:21.356911 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352348 2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 22:07:21.356911 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352350 2581 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 22:07:21.356911 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352353 2581 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 22:07:21.356911 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352356 2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 22:07:21.356911 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352358 2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 22:07:21.357405 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352361 2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 22:07:21.357405 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352364 2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 22:07:21.357405 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352366 2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 22:07:21.357405 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352368 2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 22:07:21.357405 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352371 2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 22:07:21.357405 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352373 2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 22:07:21.357405 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352378 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 22:07:21.357405 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352380 2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 22:07:21.357405 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352383 2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 22:07:21.357405 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352386 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 22:07:21.357405 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352388 2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 22:07:21.357405 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352391 2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 22:07:21.357405 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352393 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 22:07:21.357405 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352396 2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 22:07:21.357405 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352399 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 22:07:21.357405 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352401 2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 22:07:21.357405 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352404 2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 22:07:21.357405 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352406 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 22:07:21.357405 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352409 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 22:07:21.357849 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352414 2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 22:07:21.357849 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352417 2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 22:07:21.357849 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.352420 2581 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 22:07:21.357849 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354175 2581 flags.go:64] FLAG: --address="0.0.0.0" Oct 30 22:07:21.357849 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354185 2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Oct 30 22:07:21.357849 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354192 2581 flags.go:64] FLAG: --anonymous-auth="true" Oct 30 22:07:21.357849 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354197 2581 flags.go:64] FLAG: --application-metrics-count-limit="100" Oct 30 22:07:21.357849 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354202 2581 flags.go:64] FLAG: --authentication-token-webhook="false" Oct 30 22:07:21.357849 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354205 2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Oct 30 22:07:21.357849 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354210 2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Oct 30 22:07:21.357849 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354225 2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Oct 30 22:07:21.357849 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354229 2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Oct 30 22:07:21.357849 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354232 2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Oct 30 22:07:21.357849 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354236 2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Oct 30 22:07:21.357849 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354239 2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Oct 30 22:07:21.357849 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354242 2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Oct 30 22:07:21.357849 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354245 2581 flags.go:64] FLAG: --cgroup-root="" Oct 30 22:07:21.357849 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354248 2581 flags.go:64] FLAG: --cgroups-per-qos="true" Oct 30 22:07:21.357849 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354251 2581 flags.go:64] FLAG: --client-ca-file="" Oct 30 22:07:21.357849 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354255 2581 flags.go:64] FLAG: --cloud-provider="external" Oct 30 22:07:21.357849 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354259 2581 flags.go:64] FLAG: --cluster-dns="[]" Oct 30 22:07:21.357849 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354263 2581 flags.go:64] FLAG: --cluster-domain="" Oct 30 22:07:21.357849 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354265 2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Oct 30 22:07:21.358405 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354269 2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Oct 30 22:07:21.358405 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354272 2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Oct 30 22:07:21.358405 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354275 2581 flags.go:64] FLAG: --container-log-max-files="5" Oct 30 22:07:21.358405 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354279 2581 flags.go:64] FLAG: --container-log-max-size="10Mi" Oct 30 22:07:21.358405 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354283 2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Oct 30 22:07:21.358405 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354286 2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Oct 30 22:07:21.358405 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354289 2581 flags.go:64] FLAG: --containerd-namespace="k8s.io" Oct 30 22:07:21.358405 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354292 2581 flags.go:64] FLAG: --contention-profiling="false" Oct 30 22:07:21.358405 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354295 2581 flags.go:64] FLAG: --cpu-cfs-quota="true" Oct 30 22:07:21.358405 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354298 2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Oct 30 22:07:21.358405 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354301 2581 flags.go:64] FLAG: --cpu-manager-policy="none" Oct 30 22:07:21.358405 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354304 2581 flags.go:64] FLAG: --cpu-manager-policy-options="" Oct 30 22:07:21.358405 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354308 2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Oct 30 22:07:21.358405 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354311 2581 flags.go:64] FLAG: --enable-controller-attach-detach="true" Oct 30 22:07:21.358405 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354314 2581 flags.go:64] FLAG: --enable-debugging-handlers="true" Oct 30 22:07:21.358405 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354317 2581 flags.go:64] FLAG: --enable-load-reader="false" Oct 30 22:07:21.358405 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354320 2581 flags.go:64] FLAG: --enable-server="true" Oct 30 22:07:21.358405 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354324 2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Oct 30 22:07:21.358405 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354333 2581 flags.go:64] FLAG: --event-burst="100" Oct 30 22:07:21.358405 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354342 2581 flags.go:64] FLAG: --event-qps="50" Oct 30 22:07:21.358405 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354345 2581 flags.go:64] FLAG: --event-storage-age-limit="default=0" Oct 30 22:07:21.358405 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354348 2581 flags.go:64] FLAG: --event-storage-event-limit="default=0" Oct 30 22:07:21.358405 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354351 2581 flags.go:64] FLAG: --eviction-hard="" Oct 30 22:07:21.358405 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354355 2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Oct 30 22:07:21.358990 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354358 2581 flags.go:64] FLAG: --eviction-minimum-reclaim="" Oct 30 22:07:21.358990 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354361 2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Oct 30 22:07:21.358990 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354365 2581 flags.go:64] FLAG: --eviction-soft="" Oct 30 22:07:21.358990 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354368 2581 flags.go:64] FLAG: --eviction-soft-grace-period="" Oct 30 22:07:21.358990 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354371 2581 flags.go:64] FLAG: --exit-on-lock-contention="false" Oct 30 22:07:21.358990 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354374 2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Oct 30 22:07:21.358990 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354377 2581 flags.go:64] FLAG: --experimental-mounter-path="" Oct 30 22:07:21.358990 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354380 2581 flags.go:64] FLAG: --fail-cgroupv1="true" Oct 30 22:07:21.358990 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354383 2581 flags.go:64] FLAG: --fail-swap-on="true" Oct 30 22:07:21.358990 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354386 2581 flags.go:64] FLAG: --feature-gates="" Oct 30 22:07:21.358990 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354390 2581 flags.go:64] FLAG: --file-check-frequency="20s" Oct 30 22:07:21.358990 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354393 2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Oct 30 22:07:21.358990 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354397 2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Oct 30 22:07:21.358990 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354400 2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Oct 30 22:07:21.358990 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354403 2581 flags.go:64] FLAG: --healthz-port="10248" Oct 30 22:07:21.358990 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354407 2581 flags.go:64] FLAG: --help="false" Oct 30 22:07:21.358990 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354410 2581 flags.go:64] FLAG: --hostname-override="ip-10-0-142-24.ec2.internal" Oct 30 22:07:21.358990 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354413 2581 flags.go:64] FLAG: --housekeeping-interval="10s" Oct 30 22:07:21.358990 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354416 2581 flags.go:64] FLAG: --http-check-frequency="20s" Oct 30 22:07:21.358990 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354419 2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Oct 30 22:07:21.358990 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354422 2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Oct 30 22:07:21.358990 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354426 2581 flags.go:64] FLAG: --image-gc-high-threshold="85" Oct 30 22:07:21.358990 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354429 2581 flags.go:64] FLAG: --image-gc-low-threshold="80" Oct 30 22:07:21.358990 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354432 2581 flags.go:64] FLAG: --image-service-endpoint="" Oct 30 22:07:21.359582 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354435 2581 flags.go:64] FLAG: --kernel-memcg-notification="false" Oct 30 22:07:21.359582 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354438 2581 flags.go:64] FLAG: --kube-api-burst="100" Oct 30 22:07:21.359582 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354441 2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Oct 30 22:07:21.359582 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354444 2581 flags.go:64] FLAG: --kube-api-qps="50" Oct 30 22:07:21.359582 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354452 2581 flags.go:64] FLAG: --kube-reserved="" Oct 30 22:07:21.359582 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354456 2581 flags.go:64] FLAG: --kube-reserved-cgroup="" Oct 30 22:07:21.359582 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354459 2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Oct 30 22:07:21.359582 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354462 2581 flags.go:64] FLAG: --kubelet-cgroups="" Oct 30 22:07:21.359582 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354465 2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Oct 30 22:07:21.359582 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354468 2581 flags.go:64] FLAG: --lock-file="" Oct 30 22:07:21.359582 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354471 2581 flags.go:64] FLAG: --log-cadvisor-usage="false" Oct 30 22:07:21.359582 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354474 2581 flags.go:64] FLAG: --log-flush-frequency="5s" Oct 30 22:07:21.359582 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354477 2581 flags.go:64] FLAG: --log-json-info-buffer-size="0" Oct 30 22:07:21.359582 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354482 2581 flags.go:64] FLAG: --log-json-split-stream="false" Oct 30 22:07:21.359582 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354485 2581 flags.go:64] FLAG: --log-text-info-buffer-size="0" Oct 30 22:07:21.359582 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354488 2581 flags.go:64] FLAG: --log-text-split-stream="false" Oct 30 22:07:21.359582 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354491 2581 flags.go:64] FLAG: --logging-format="text" Oct 30 22:07:21.359582 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354494 2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Oct 30 22:07:21.359582 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354498 2581 flags.go:64] FLAG: --make-iptables-util-chains="true" Oct 30 22:07:21.359582 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354500 2581 flags.go:64] FLAG: --manifest-url="" Oct 30 22:07:21.359582 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354503 2581 flags.go:64] FLAG: --manifest-url-header="" Oct 30 22:07:21.359582 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354509 2581 flags.go:64] FLAG: --max-housekeeping-interval="15s" Oct 30 22:07:21.359582 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354512 2581 flags.go:64] FLAG: --max-open-files="1000000" Oct 30 22:07:21.359582 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354515 2581 flags.go:64] FLAG: --max-pods="110" Oct 30 22:07:21.359582 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354518 2581 flags.go:64] FLAG: --maximum-dead-containers="-1" Oct 30 22:07:21.360156 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354522 2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Oct 30 22:07:21.360156 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354525 2581 flags.go:64] FLAG: --memory-manager-policy="None" Oct 30 22:07:21.360156 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354528 2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Oct 30 22:07:21.360156 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354531 2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Oct 30 22:07:21.360156 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354534 2581 flags.go:64] FLAG: --node-ip="0.0.0.0" Oct 30 22:07:21.360156 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354537 2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Oct 30 22:07:21.360156 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354544 2581 flags.go:64] FLAG: --node-status-max-images="50" Oct 30 22:07:21.360156 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354547 2581 flags.go:64] FLAG: --node-status-update-frequency="10s" Oct 30 22:07:21.360156 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354550 2581 flags.go:64] FLAG: --oom-score-adj="-999" Oct 30 22:07:21.360156 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354553 2581 flags.go:64] FLAG: --pod-cidr="" Oct 30 22:07:21.360156 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354556 2581 flags.go:64] FLAG: --pod-manifest-path="" Oct 30 22:07:21.360156 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354559 2581 flags.go:64] FLAG: --pod-max-pids="-1" Oct 30 22:07:21.360156 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354568 2581 flags.go:64] FLAG: --pods-per-core="0" Oct 30 22:07:21.360156 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354571 2581 flags.go:64] FLAG: --port="10250" Oct 30 22:07:21.360156 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354574 2581 flags.go:64] FLAG: --protect-kernel-defaults="false" Oct 30 22:07:21.360156 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354577 2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f54b1358f274f8fa" Oct 30 22:07:21.360156 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354580 2581 flags.go:64] FLAG: --qos-reserved="" Oct 30 22:07:21.360156 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354584 2581 flags.go:64] FLAG: --read-only-port="10255" Oct 30 22:07:21.360156 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354587 2581 flags.go:64] FLAG: --register-node="true" Oct 30 22:07:21.360156 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354590 2581 flags.go:64] FLAG: --register-with-taints="" Oct 30 22:07:21.360156 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354594 2581 flags.go:64] FLAG: --registry-burst="10" Oct 30 22:07:21.360156 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354598 2581 flags.go:64] FLAG: --registry-qps="5" Oct 30 22:07:21.360156 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354601 2581 flags.go:64] FLAG: --reserved-cpus="" Oct 30 22:07:21.360156 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354604 2581 flags.go:64] FLAG: --reserved-memory="" Oct 30 22:07:21.360156 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354608 2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Oct 30 22:07:21.360156 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354611 2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Oct 30 22:07:21.360812 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354614 2581 flags.go:64] FLAG: --rotate-certificates="false" Oct 30 22:07:21.360812 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354617 2581 flags.go:64] FLAG: --rotate-server-certificates="false" Oct 30 22:07:21.360812 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354620 2581 flags.go:64] FLAG: --runonce="false" Oct 30 22:07:21.360812 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354623 2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Oct 30 22:07:21.360812 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354626 2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Oct 30 22:07:21.360812 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354629 2581 flags.go:64] FLAG: --seccomp-default="false" Oct 30 22:07:21.360812 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354632 2581 flags.go:64] FLAG: --serialize-image-pulls="true" Oct 30 22:07:21.360812 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354635 2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Oct 30 22:07:21.360812 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354638 2581 flags.go:64] FLAG: --storage-driver-db="cadvisor" Oct 30 22:07:21.360812 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354641 2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Oct 30 22:07:21.360812 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354644 2581 flags.go:64] FLAG: --storage-driver-password="root" Oct 30 22:07:21.360812 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354647 2581 flags.go:64] FLAG: --storage-driver-secure="false" Oct 30 22:07:21.360812 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354650 2581 flags.go:64] FLAG: --storage-driver-table="stats" Oct 30 22:07:21.360812 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354653 2581 flags.go:64] FLAG: --storage-driver-user="root" Oct 30 22:07:21.360812 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354656 2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Oct 30 22:07:21.360812 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354659 2581 flags.go:64] FLAG: --sync-frequency="1m0s" Oct 30 22:07:21.360812 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354662 2581 flags.go:64] FLAG: --system-cgroups="" Oct 30 22:07:21.360812 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354665 2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Oct 30 22:07:21.360812 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354674 2581 flags.go:64] FLAG: --system-reserved-cgroup="" Oct 30 22:07:21.360812 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354685 2581 flags.go:64] FLAG: --tls-cert-file="" Oct 30 22:07:21.360812 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354688 2581 flags.go:64] FLAG: --tls-cipher-suites="[]" Oct 30 22:07:21.360812 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354694 2581 flags.go:64] FLAG: --tls-min-version="" Oct 30 22:07:21.360812 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354697 2581 flags.go:64] FLAG: --tls-private-key-file="" Oct 30 22:07:21.360812 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354702 2581 flags.go:64] FLAG: --topology-manager-policy="none" Oct 30 22:07:21.360812 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354705 2581 flags.go:64] FLAG: --topology-manager-policy-options="" Oct 30 22:07:21.361516 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354708 2581 flags.go:64] FLAG: --topology-manager-scope="container" Oct 30 22:07:21.361516 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354711 2581 flags.go:64] FLAG: --v="2" Oct 30 22:07:21.361516 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354715 2581 flags.go:64] FLAG: --version="false" Oct 30 22:07:21.361516 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354720 2581 flags.go:64] FLAG: --vmodule="" Oct 30 22:07:21.361516 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354724 2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Oct 30 22:07:21.361516 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.354727 2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Oct 30 22:07:21.361516 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.354933 2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 22:07:21.361516 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.354936 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 22:07:21.361516 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.354939 2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 22:07:21.361516 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.354942 2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 22:07:21.361516 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.354945 2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 22:07:21.361516 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.354947 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 22:07:21.361516 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.354950 2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 22:07:21.361516 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.354953 2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 22:07:21.361516 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.354955 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 22:07:21.361516 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.354958 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 22:07:21.361516 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.354961 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 22:07:21.361516 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.354964 2581 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 22:07:21.361516 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.354966 2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 22:07:21.361516 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.354969 2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 22:07:21.361516 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.354971 2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 22:07:21.362326 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.354974 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 22:07:21.362326 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.354976 2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 22:07:21.362326 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.354979 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 22:07:21.362326 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.354982 2581 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 22:07:21.362326 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.354985 2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 22:07:21.362326 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.354988 2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 22:07:21.362326 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.354997 2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 22:07:21.362326 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355000 2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 22:07:21.362326 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355002 2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 22:07:21.362326 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355007 2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 22:07:21.362326 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355010 2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 22:07:21.362326 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355012 2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 22:07:21.362326 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355015 2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 22:07:21.362326 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355018 2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 22:07:21.362326 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355020 2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 22:07:21.362326 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355023 2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 22:07:21.362326 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355027 2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 22:07:21.362326 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355031 2581 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 22:07:21.362326 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355034 2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 22:07:21.362326 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355037 2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 22:07:21.362828 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355039 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 22:07:21.362828 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355043 2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 22:07:21.362828 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355047 2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 22:07:21.362828 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355049 2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 22:07:21.362828 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355052 2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 22:07:21.362828 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355055 2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 22:07:21.362828 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355058 2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 22:07:21.362828 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355061 2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 22:07:21.362828 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355064 2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 22:07:21.362828 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355066 2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 22:07:21.362828 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355069 2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 22:07:21.362828 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355072 2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 22:07:21.362828 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355074 2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 22:07:21.362828 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355077 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 22:07:21.362828 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355080 2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 22:07:21.362828 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355082 2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 22:07:21.362828 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355086 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 22:07:21.362828 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355089 2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 22:07:21.362828 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355092 2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 22:07:21.363291 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355100 2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 22:07:21.363291 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355103 2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 22:07:21.363291 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355107 2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 22:07:21.363291 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355109 2581 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 22:07:21.363291 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355112 2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 22:07:21.363291 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355115 2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 22:07:21.363291 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355117 2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 22:07:21.363291 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355120 2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 22:07:21.363291 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355123 2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 22:07:21.363291 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355125 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 22:07:21.363291 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355128 2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 22:07:21.363291 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355130 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 22:07:21.363291 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355133 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 22:07:21.363291 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355136 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 22:07:21.363291 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355138 2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 22:07:21.363291 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355141 2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 22:07:21.363291 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355149 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 22:07:21.363291 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355152 2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 22:07:21.363291 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355154 2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 22:07:21.363734 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355157 2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 22:07:21.363734 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355160 2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 22:07:21.363734 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355163 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 22:07:21.363734 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355165 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 22:07:21.363734 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355168 2581 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 22:07:21.363734 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355170 2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 22:07:21.363734 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355173 2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 22:07:21.363734 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355175 2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 22:07:21.363734 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355178 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 22:07:21.363734 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355180 2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 22:07:21.363734 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355185 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 22:07:21.363734 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355188 2581 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 22:07:21.363734 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355190 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 22:07:21.363734 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355193 2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 22:07:21.363734 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355203 2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 22:07:21.363734 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355207 2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 22:07:21.363734 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355210 2581 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 22:07:21.363734 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355213 2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 22:07:21.363734 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355227 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 22:07:21.364179 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355230 2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 22:07:21.364179 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355233 2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 22:07:21.364179 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355235 2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 22:07:21.364179 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355238 2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 22:07:21.364179 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355240 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 22:07:21.364179 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355243 2581 feature_gate.go:386] unrecognized feature gate: Example Oct 30 22:07:21.364179 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355246 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 22:07:21.364179 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355248 2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 22:07:21.364179 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355251 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 22:07:21.364179 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355253 2581 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 22:07:21.364179 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355256 2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 22:07:21.364179 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355258 2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 22:07:21.364179 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355262 2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 22:07:21.364179 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355266 2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 22:07:21.364179 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.355295 2581 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 22:07:21.364563 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355341 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 22:07:21.364563 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355346 2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 22:07:21.364563 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355349 2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 22:07:21.364563 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355353 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 22:07:21.364563 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355356 2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 22:07:21.364563 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355359 2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 22:07:21.364563 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355361 2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 22:07:21.364563 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355364 2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 22:07:21.364563 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355366 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 22:07:21.364563 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355371 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 22:07:21.364563 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355373 2581 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 22:07:21.364563 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355376 2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 22:07:21.364563 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355378 2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 22:07:21.364563 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355388 2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 22:07:21.364563 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355391 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 22:07:21.364563 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355394 2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 22:07:21.364563 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355396 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 22:07:21.364563 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355399 2581 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 22:07:21.364563 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355402 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 22:07:21.365012 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355405 2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 22:07:21.365012 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355407 2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 22:07:21.365012 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355410 2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 22:07:21.365012 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355412 2581 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 22:07:21.365012 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355415 2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 22:07:21.365012 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355418 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 22:07:21.365012 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355420 2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 22:07:21.365012 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355423 2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 22:07:21.365012 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355425 2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 22:07:21.365012 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355428 2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 22:07:21.365012 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355431 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 22:07:21.365012 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355433 2581 feature_gate.go:386] unrecognized feature gate: Example Oct 30 22:07:21.365012 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355436 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 22:07:21.365012 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355438 2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 22:07:21.365012 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355441 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 22:07:21.365012 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355443 2581 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 22:07:21.365012 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355446 2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 22:07:21.365012 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355448 2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 22:07:21.365012 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355452 2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 22:07:21.365546 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355455 2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 22:07:21.365546 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355458 2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 22:07:21.365546 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355461 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 22:07:21.365546 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355465 2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 22:07:21.365546 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355468 2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 22:07:21.365546 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355470 2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 22:07:21.365546 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355473 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 22:07:21.365546 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355477 2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 22:07:21.365546 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355486 2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 22:07:21.365546 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355489 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 22:07:21.365546 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355491 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 22:07:21.365546 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355494 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 22:07:21.365546 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355497 2581 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 22:07:21.365546 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355499 2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 22:07:21.365546 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355502 2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 22:07:21.365546 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355504 2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 22:07:21.365546 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355507 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 22:07:21.365546 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355509 2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 22:07:21.365546 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355512 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 22:07:21.365546 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355515 2581 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 22:07:21.366026 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355517 2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 22:07:21.366026 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355520 2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 22:07:21.366026 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355522 2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 22:07:21.366026 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355525 2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 22:07:21.366026 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355527 2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 22:07:21.366026 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355530 2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 22:07:21.366026 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355533 2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 22:07:21.366026 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355535 2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 22:07:21.366026 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355538 2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 22:07:21.366026 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355541 2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 22:07:21.366026 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355544 2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 22:07:21.366026 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355546 2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 22:07:21.366026 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355550 2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 22:07:21.366026 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355554 2581 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 22:07:21.366026 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355557 2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 22:07:21.366026 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355561 2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 22:07:21.366026 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355564 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 22:07:21.366026 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355567 2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 22:07:21.366026 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355570 2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 22:07:21.366026 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355573 2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 22:07:21.366555 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355575 2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 22:07:21.366555 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355584 2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 22:07:21.366555 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355586 2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 22:07:21.366555 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355589 2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 22:07:21.366555 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355592 2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 22:07:21.366555 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355594 2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 22:07:21.366555 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355597 2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 22:07:21.366555 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355600 2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 22:07:21.366555 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355602 2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 22:07:21.366555 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355605 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 22:07:21.366555 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355608 2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 22:07:21.366555 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355610 2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 22:07:21.366555 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355613 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 22:07:21.366555 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355615 2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 22:07:21.366555 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355618 2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 22:07:21.366555 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355620 2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 22:07:21.366555 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355623 2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 22:07:21.366555 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355626 2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 22:07:21.366555 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355628 2581 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 22:07:21.366555 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355631 2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 22:07:21.367030 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355633 2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 22:07:21.367030 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355636 2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 22:07:21.367030 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355638 2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 22:07:21.367030 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355641 2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 22:07:21.367030 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355643 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 22:07:21.367030 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355646 2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 22:07:21.367030 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355649 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 22:07:21.367030 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.355653 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 22:07:21.367030 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.363776 2581 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Oct 30 22:07:21.367030 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.363792 2581 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Oct 30 22:07:21.367030 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.364407 2581 watchdog_linux.go:95] "Systemd watchdog is not enabled" Oct 30 22:07:21.367030 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.364414 2581 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Oct 30 22:07:21.367030 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364473 2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 22:07:21.367030 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364478 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 22:07:21.367030 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364482 2581 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 22:07:21.367030 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364485 2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 22:07:21.367030 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364488 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 22:07:21.367030 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364490 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 22:07:21.367030 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364493 2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 22:07:21.367503 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364496 2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 22:07:21.367503 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364499 2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 22:07:21.367503 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364501 2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 22:07:21.367503 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364504 2581 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 22:07:21.367503 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364506 2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 22:07:21.367503 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364509 2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 22:07:21.367503 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364512 2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 22:07:21.367503 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364514 2581 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 22:07:21.367503 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364517 2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 22:07:21.367503 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364520 2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 22:07:21.367503 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364522 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 22:07:21.367503 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364525 2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 22:07:21.367503 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364528 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 22:07:21.367503 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364530 2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 22:07:21.367503 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364533 2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 22:07:21.367503 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364536 2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 22:07:21.367503 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364539 2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 22:07:21.367503 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364542 2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 22:07:21.367503 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364544 2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 22:07:21.367503 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364547 2581 feature_gate.go:386] unrecognized feature gate: Example Oct 30 22:07:21.368020 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364549 2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 22:07:21.368020 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364552 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 22:07:21.368020 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364555 2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 22:07:21.368020 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364560 2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 22:07:21.368020 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364564 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 22:07:21.368020 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364568 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 22:07:21.368020 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364571 2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 22:07:21.368020 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364574 2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 22:07:21.368020 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364577 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 22:07:21.368020 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364579 2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 22:07:21.368020 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364582 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 22:07:21.368020 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364585 2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 22:07:21.368020 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364588 2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 22:07:21.368020 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364590 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 22:07:21.368020 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364593 2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 22:07:21.368020 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364595 2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 22:07:21.368020 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364598 2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 22:07:21.368020 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364602 2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 22:07:21.368020 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364606 2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 22:07:21.368488 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364609 2581 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 22:07:21.368488 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364612 2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 22:07:21.368488 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364615 2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 22:07:21.368488 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364618 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 22:07:21.368488 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364621 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 22:07:21.368488 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364625 2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 22:07:21.368488 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364627 2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 22:07:21.368488 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364630 2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 22:07:21.368488 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364633 2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 22:07:21.368488 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364636 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 22:07:21.368488 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364639 2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 22:07:21.368488 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364641 2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 22:07:21.368488 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364644 2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 22:07:21.368488 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364647 2581 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 22:07:21.368488 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364650 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 22:07:21.368488 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364653 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 22:07:21.368488 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364656 2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 22:07:21.368488 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364658 2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 22:07:21.368488 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364661 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 22:07:21.368488 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364663 2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 22:07:21.369099 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364666 2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 22:07:21.369099 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364668 2581 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 22:07:21.369099 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364671 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 22:07:21.369099 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364673 2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 22:07:21.369099 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364676 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 22:07:21.369099 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364679 2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 22:07:21.369099 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364681 2581 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 22:07:21.369099 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364683 2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 22:07:21.369099 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364686 2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 22:07:21.369099 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364689 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 22:07:21.369099 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364692 2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 22:07:21.369099 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364694 2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 22:07:21.369099 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364696 2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 22:07:21.369099 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364700 2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 22:07:21.369099 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364702 2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 22:07:21.369099 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364705 2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 22:07:21.369099 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364707 2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 22:07:21.369099 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364711 2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 22:07:21.369099 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364713 2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 22:07:21.369099 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364716 2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 22:07:21.369762 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364718 2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 22:07:21.369762 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364721 2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 22:07:21.369762 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364723 2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 22:07:21.369762 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364726 2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 22:07:21.369762 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364728 2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 22:07:21.369762 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364731 2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 22:07:21.369762 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364733 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 22:07:21.369762 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364736 2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 22:07:21.369762 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364738 2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 22:07:21.369762 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364741 2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 22:07:21.369762 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364743 2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 22:07:21.369762 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364746 2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 22:07:21.369762 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364748 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 22:07:21.369762 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364751 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 22:07:21.369762 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364753 2581 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 22:07:21.369762 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364756 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 22:07:21.369762 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364759 2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 22:07:21.369762 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364761 2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 22:07:21.369762 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364763 2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 22:07:21.370229 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364766 2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 22:07:21.370229 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.364794 2581 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 22:07:21.370229 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364932 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 22:07:21.370229 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364936 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 22:07:21.370229 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364939 2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 22:07:21.370229 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364942 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 22:07:21.370229 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364944 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 22:07:21.370229 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364947 2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 22:07:21.370229 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364950 2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 22:07:21.370229 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364953 2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 22:07:21.370229 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364955 2581 feature_gate.go:386] unrecognized feature gate: Example Oct 30 22:07:21.370229 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364958 2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 22:07:21.370229 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364961 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 22:07:21.370229 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364965 2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 22:07:21.370229 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364969 2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 22:07:21.370588 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364972 2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 22:07:21.370588 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364975 2581 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 22:07:21.370588 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364978 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 22:07:21.370588 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364980 2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 22:07:21.370588 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364983 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 22:07:21.370588 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364985 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 22:07:21.370588 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364988 2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 22:07:21.370588 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364991 2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 22:07:21.370588 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364994 2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 22:07:21.370588 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.364998 2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 22:07:21.370588 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365001 2581 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 22:07:21.370588 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365004 2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 22:07:21.370588 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365008 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 22:07:21.370588 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365011 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 22:07:21.370588 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365014 2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 22:07:21.370588 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365017 2581 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 22:07:21.370588 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365020 2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 22:07:21.370588 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365023 2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 22:07:21.370588 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365025 2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 22:07:21.371074 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365028 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 22:07:21.371074 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365030 2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 22:07:21.371074 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365033 2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 22:07:21.371074 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365035 2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 22:07:21.371074 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365038 2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 22:07:21.371074 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365040 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 22:07:21.371074 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365043 2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 22:07:21.371074 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365046 2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 22:07:21.371074 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365048 2581 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 22:07:21.371074 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365051 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 22:07:21.371074 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365054 2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 22:07:21.371074 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365056 2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 22:07:21.371074 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365059 2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 22:07:21.371074 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365061 2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 22:07:21.371074 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365064 2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 22:07:21.371074 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365067 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 22:07:21.371074 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365069 2581 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 22:07:21.371074 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365071 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 22:07:21.371074 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365074 2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 22:07:21.371550 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365077 2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 22:07:21.371550 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365079 2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 22:07:21.371550 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365081 2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 22:07:21.371550 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365084 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 22:07:21.371550 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365087 2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 22:07:21.371550 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365096 2581 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 22:07:21.371550 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365100 2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 22:07:21.371550 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365102 2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 22:07:21.371550 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365105 2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 22:07:21.371550 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365107 2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 22:07:21.371550 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365110 2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 22:07:21.371550 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365112 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 22:07:21.371550 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365115 2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 22:07:21.371550 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365117 2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 22:07:21.371550 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365120 2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 22:07:21.371550 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365122 2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 22:07:21.371550 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365125 2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 22:07:21.371550 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365127 2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 22:07:21.371550 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365130 2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 22:07:21.371550 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365132 2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 22:07:21.372020 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365136 2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 22:07:21.372020 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365139 2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 22:07:21.372020 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365142 2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 22:07:21.372020 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365144 2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 22:07:21.372020 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365147 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 22:07:21.372020 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365150 2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 22:07:21.372020 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365152 2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 22:07:21.372020 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365155 2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 22:07:21.372020 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365157 2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 22:07:21.372020 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365160 2581 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 22:07:21.372020 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365162 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 22:07:21.372020 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365165 2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 22:07:21.372020 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365167 2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 22:07:21.372020 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365170 2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 22:07:21.372020 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365172 2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 22:07:21.372020 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365175 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 22:07:21.372020 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365178 2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 22:07:21.372020 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365180 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 22:07:21.372020 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365183 2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 22:07:21.372020 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365186 2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 22:07:21.372520 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365188 2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 22:07:21.372520 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365191 2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 22:07:21.372520 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365193 2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 22:07:21.372520 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365196 2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 22:07:21.372520 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365198 2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 22:07:21.372520 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365201 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 22:07:21.372520 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365203 2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 22:07:21.372520 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365206 2581 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 22:07:21.372520 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365209 2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 22:07:21.372520 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365211 2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 22:07:21.372520 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365227 2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 22:07:21.372520 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365232 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 22:07:21.372520 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365234 2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 22:07:21.372520 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365238 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 22:07:21.372520 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365241 2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 22:07:21.372520 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.365265 2581 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 22:07:21.372969 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365298 2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 22:07:21.372969 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365302 2581 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 22:07:21.372969 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365305 2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 22:07:21.372969 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365308 2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 22:07:21.372969 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365310 2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 22:07:21.372969 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365313 2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 22:07:21.372969 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365315 2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 22:07:21.372969 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365318 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 22:07:21.372969 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365321 2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 22:07:21.372969 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365323 2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 22:07:21.372969 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365326 2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 22:07:21.372969 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365328 2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 22:07:21.372969 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365331 2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 22:07:21.372969 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365333 2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 22:07:21.372969 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365336 2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 22:07:21.372969 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365338 2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 22:07:21.372969 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365341 2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 22:07:21.372969 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365343 2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 22:07:21.372969 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365346 2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 22:07:21.372969 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365348 2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 22:07:21.373602 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365351 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 22:07:21.373602 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365354 2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 22:07:21.373602 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365356 2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 22:07:21.373602 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365359 2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 22:07:21.373602 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365361 2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 22:07:21.373602 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365364 2581 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 22:07:21.373602 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365366 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 22:07:21.373602 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365369 2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 22:07:21.373602 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365371 2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 22:07:21.373602 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365374 2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 22:07:21.373602 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365376 2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 22:07:21.373602 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365379 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 22:07:21.373602 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365382 2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 22:07:21.373602 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365384 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 22:07:21.373602 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365387 2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 22:07:21.373602 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365390 2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 22:07:21.373602 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365392 2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 22:07:21.373602 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365395 2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 22:07:21.373602 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365397 2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 22:07:21.373602 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365400 2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 22:07:21.374442 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365403 2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 22:07:21.374442 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365405 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 22:07:21.374442 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365408 2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 22:07:21.374442 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365411 2581 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 22:07:21.374442 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365413 2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 22:07:21.374442 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365416 2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 22:07:21.374442 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365418 2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 22:07:21.374442 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365420 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 22:07:21.374442 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365423 2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 22:07:21.374442 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365425 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 22:07:21.374442 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365428 2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 22:07:21.374442 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365430 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 22:07:21.374442 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365433 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 22:07:21.374442 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365436 2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 22:07:21.374442 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365438 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 22:07:21.374442 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365443 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 22:07:21.374442 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365446 2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 22:07:21.374442 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365448 2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 22:07:21.374442 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365451 2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 22:07:21.374965 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365454 2581 feature_gate.go:386] unrecognized feature gate: Example Oct 30 22:07:21.374965 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365456 2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 22:07:21.374965 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365459 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 22:07:21.374965 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365463 2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 22:07:21.374965 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365467 2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 22:07:21.374965 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365471 2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 22:07:21.374965 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365473 2581 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 22:07:21.374965 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365476 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 22:07:21.374965 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365478 2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 22:07:21.374965 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365481 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 22:07:21.374965 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365485 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 22:07:21.374965 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365488 2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 22:07:21.374965 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365491 2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 22:07:21.374965 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365494 2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 22:07:21.374965 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365498 2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 22:07:21.374965 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365501 2581 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 22:07:21.374965 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365504 2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 22:07:21.374965 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365507 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 22:07:21.374965 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365510 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 22:07:21.375550 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365512 2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 22:07:21.375550 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365515 2581 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 22:07:21.375550 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365517 2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 22:07:21.375550 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365520 2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 22:07:21.375550 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365522 2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 22:07:21.375550 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365525 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 22:07:21.375550 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365527 2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 22:07:21.375550 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365530 2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 22:07:21.375550 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365533 2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 22:07:21.375550 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365536 2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 22:07:21.375550 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365539 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 22:07:21.375550 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365542 2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 22:07:21.375550 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365544 2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 22:07:21.375550 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365547 2581 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 22:07:21.375550 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365550 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 22:07:21.375550 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365553 2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 22:07:21.375550 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365555 2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 22:07:21.375550 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365559 2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 22:07:21.375550 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365562 2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 22:07:21.375550 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365565 2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 22:07:21.376085 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365568 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 22:07:21.376085 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365570 2581 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 22:07:21.376085 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365573 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 22:07:21.376085 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365576 2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 22:07:21.376085 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365579 2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 22:07:21.376085 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365581 2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 22:07:21.376085 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365584 2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 22:07:21.376085 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:21.365587 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 22:07:21.376085 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.365662 2581 server.go:957] "Client rotation is on, will bootstrap in background" Oct 30 22:07:21.376085 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.367677 2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Oct 30 22:07:21.376085 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.368587 2581 server.go:1014] "Starting client certificate rotation" Oct 30 22:07:21.376085 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.368690 2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 30 22:07:21.376085 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.368735 2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 30 22:07:21.376085 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.370554 2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 22:07:21.376085 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.374600 2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 22:07:21.384400 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.384384 2581 log.go:25] "Validated CRI v1 runtime API" Oct 30 22:07:21.387677 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.387662 2581 log.go:25] "Validated CRI v1 image API" Oct 30 22:07:21.388478 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.388448 2581 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Oct 30 22:07:21.391640 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.391621 2581 fs.go:135] Filesystem UUIDs: map[34377297-fbd4-4393-8bed-a10a13622c62:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 7ddc0a6e-163a-46a0-824c-88dd7e6fd7ac:/dev/nvme0n1p4] Oct 30 22:07:21.391705 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.391640 2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}] Oct 30 22:07:21.397725 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.397612 2581 manager.go:217] Machine: {Timestamp:2026-10-30 22:07:21.396298059 +0000 UTC m=+0.322400377 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102328 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e72500166de0eca2c19f9ae79a998 SystemUUID:ec2e7250-0166-de0e-ca2c-19f9ae79a998 BootID:f366a10e-7b18-4883-b9c8-d788d9a7e2ec Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:f6:06:9a:19 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:f6:06:9a:19 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ee:3d:6f:e9:85:16 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Oct 30 22:07:21.398236 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.398212 2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Oct 30 22:07:21.398318 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.398306 2581 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Oct 30 22:07:21.401863 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.401833 2581 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Oct 30 22:07:21.401988 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.401865 2581 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-24.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Oct 30 22:07:21.402044 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.401990 2581 topology_manager.go:143] "Creating topology manager with none policy" Oct 30 22:07:21.402044 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.401999 2581 container_manager_linux.go:311] "Creating device plugin manager" Oct 30 22:07:21.402044 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.402040 2581 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 22:07:21.402159 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.402057 2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 22:07:21.402799 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.402788 2581 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Oct 30 22:07:21.406603 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.406590 2581 state_mem.go:41] "Initialized" logger="CPUManager state memory" Oct 30 22:07:21.406723 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.406713 2581 server.go:1235] "Using root directory" path="/var/lib/kubelet" Oct 30 22:07:21.407350 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.407340 2581 kubelet.go:493] "Attempting to sync node with API server" Oct 30 22:07:21.407380 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.407354 2581 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Oct 30 22:07:21.407380 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.407365 2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Oct 30 22:07:21.407380 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.407373 2581 kubelet.go:405] "Adding apiserver pod source" Oct 30 22:07:21.407380 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.407381 2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Oct 30 22:07:21.407708 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.407696 2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 22:07:21.409979 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.409948 2581 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Oct 30 22:07:21.415198 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.415185 2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 22:07:21.415960 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.415948 2581 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Oct 30 22:07:21.415998 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.415985 2581 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Oct 30 22:07:21.416144 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.416131 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Oct 30 22:07:21.416173 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.416161 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Oct 30 22:07:21.416199 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.416176 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Oct 30 22:07:21.416199 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.416190 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Oct 30 22:07:21.416267 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.416202 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Oct 30 22:07:21.416267 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.416231 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Oct 30 22:07:21.416876 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.416866 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Oct 30 22:07:21.416908 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.416877 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Oct 30 22:07:21.416908 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.416885 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Oct 30 22:07:21.416908 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.416891 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Oct 30 22:07:21.416908 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.416899 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Oct 30 22:07:21.416908 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.416907 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Oct 30 22:07:21.417607 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.417597 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Oct 30 22:07:21.417645 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.417610 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Oct 30 22:07:21.421287 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.421275 2581 server.go:1263] "Started kubelet" Oct 30 22:07:21.421404 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.421373 2581 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Oct 30 22:07:21.421438 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.421365 2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Oct 30 22:07:21.421466 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.421440 2581 server_v1.go:49] "podresources" method="list" useActivePods=true Oct 30 22:07:21.421996 ip-10-0-142-24 systemd[1]: Started Kubernetes Kubelet. Oct 30 22:07:21.422994 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.422518 2581 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Oct 30 22:07:21.424010 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.423995 2581 server.go:317] "Adding debug handlers to kubelet server" Oct 30 22:07:21.428202 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:21.428181 2581 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Oct 30 22:07:21.428503 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.428483 2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Oct 30 22:07:21.429079 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.429063 2581 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Oct 30 22:07:21.429194 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:21.429175 2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-24.ec2.internal\" not found" Oct 30 22:07:21.429738 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.429721 2581 volume_manager.go:309] "The desired_state_of_world populator starts" Oct 30 22:07:21.429809 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.429741 2581 volume_manager.go:311] "Starting Kubelet Volume Manager" Oct 30 22:07:21.429850 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.429825 2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Oct 30 22:07:21.429891 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.429880 2581 reconstruct.go:97] "Volume reconstruction finished" Oct 30 22:07:21.429891 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.429890 2581 reconciler.go:29] "Reconciler: start to sync state" Oct 30 22:07:21.430247 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.430211 2581 factory.go:153] Registering CRI-O factory Oct 30 22:07:21.430316 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.430253 2581 factory.go:223] Registration of the crio container factory successfully Oct 30 22:07:21.430316 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.430304 2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Oct 30 22:07:21.430316 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.430315 2581 factory.go:55] Registering systemd factory Oct 30 22:07:21.430425 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.430322 2581 factory.go:223] Registration of the systemd container factory successfully Oct 30 22:07:21.430425 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.430339 2581 factory.go:103] Registering Raw factory Oct 30 22:07:21.430425 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.430354 2581 manager.go:1199] Started watching for new ooms in manager Oct 30 22:07:21.431306 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.431291 2581 manager.go:319] Starting recovery of all containers Oct 30 22:07:21.443533 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.443515 2581 manager.go:324] Recovery completed Oct 30 22:07:21.447805 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.447792 2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 22:07:21.449469 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.449450 2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-24.ec2.internal" event="NodeHasSufficientMemory" Oct 30 22:07:21.449576 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.449495 2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-24.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 22:07:21.449576 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.449512 2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-24.ec2.internal" event="NodeHasSufficientPID" Oct 30 22:07:21.450034 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.450019 2581 cpu_manager.go:226] "Starting" policy="none" Oct 30 22:07:21.450034 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.450028 2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 22:07:21.450034 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.450033 2581 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Oct 30 22:07:21.450162 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.450052 2581 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Oct 30 22:07:21.450673 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.450660 2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-24.ec2.internal" event="NodeHasSufficientMemory" Oct 30 22:07:21.450724 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.450686 2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-24.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 22:07:21.450724 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.450700 2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-24.ec2.internal" event="NodeHasSufficientPID" Oct 30 22:07:21.452334 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.452322 2581 policy_none.go:50] "Start" Oct 30 22:07:21.452383 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.452337 2581 memory_manager.go:187] "Starting memorymanager" policy="None" Oct 30 22:07:21.452383 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.452347 2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Oct 30 22:07:21.454452 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.454440 2581 policy_none.go:44] "Start" Oct 30 22:07:21.456816 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.456799 2581 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Oct 30 22:07:21.456886 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.456832 2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xrtqd" Oct 30 22:07:21.459201 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:21.459150 2581 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-24.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Oct 30 22:07:21.459273 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.459203 2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-24.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Oct 30 22:07:21.460109 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:21.459160 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-24.ec2.internal.18e36e044fcb733f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-24.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-24.ec2.internal,},FirstTimestamp:2026-10-30 22:07:21.421247295 +0000 UTC m=+0.347349611,LastTimestamp:2026-10-30 22:07:21.421247295 +0000 UTC m=+0.347349611,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-24.ec2.internal,}" Oct 30 22:07:21.469464 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.469445 2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xrtqd" Oct 30 22:07:21.475575 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:21.475483 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-24.ec2.internal.18e36e04517a1cff default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-24.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-24.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-24.ec2.internal,},FirstTimestamp:2026-10-30 22:07:21.449471231 +0000 UTC m=+0.375573552,LastTimestamp:2026-10-30 22:07:21.449471231 +0000 UTC m=+0.375573552,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-24.ec2.internal,}" Oct 30 22:07:21.475765 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:21.475736 2581 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" Oct 30 22:07:21.493806 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.493779 2581 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Oct 30 22:07:21.496463 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.496451 2581 manager.go:341] "Starting Device Plugin manager" Oct 30 22:07:21.496517 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:21.496482 2581 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Oct 30 22:07:21.496517 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.496490 2581 server.go:87] "Starting device plugin registration server" Oct 30 22:07:21.496732 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.496720 2581 eviction_manager.go:194] "Eviction manager: starting control loop" Oct 30 22:07:21.496782 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.496734 2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Oct 30 22:07:21.496855 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.496840 2581 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Oct 30 22:07:21.497004 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.496993 2581 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Oct 30 22:07:21.497101 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.497085 2581 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Oct 30 22:07:21.497409 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:21.497388 2581 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Oct 30 22:07:21.497485 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:21.497437 2581 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-24.ec2.internal\" not found" Oct 30 22:07:21.507876 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.507859 2581 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 22:07:21.530135 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.529848 2581 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 22:07:21.560641 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.560617 2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Oct 30 22:07:21.561768 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.561746 2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Oct 30 22:07:21.561768 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.561768 2581 status_manager.go:249] "Starting to sync pod status with apiserver" Oct 30 22:07:21.561889 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.561786 2581 kubelet.go:2521] "Starting kubelet main sync loop" Oct 30 22:07:21.561889 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:21.561818 2581 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Oct 30 22:07:21.577722 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.577706 2581 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 22:07:21.597546 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.597508 2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 22:07:21.598331 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.598315 2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-24.ec2.internal" event="NodeHasSufficientMemory" Oct 30 22:07:21.598417 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.598354 2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-24.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 22:07:21.598417 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.598369 2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-24.ec2.internal" event="NodeHasSufficientPID" Oct 30 22:07:21.598417 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.598399 2581 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-142-24.ec2.internal" Oct 30 22:07:21.618111 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.617959 2581 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-142-24.ec2.internal" Oct 30 22:07:21.662524 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.662502 2581 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-24.ec2.internal"] Oct 30 22:07:21.664796 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.664781 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal" Oct 30 22:07:21.666540 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.666526 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-24.ec2.internal" Oct 30 22:07:21.686414 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.686400 2581 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal" Oct 30 22:07:21.688350 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.688335 2581 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-24.ec2.internal" Oct 30 22:07:21.711341 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.711321 2581 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 22:07:21.714064 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.714051 2581 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 22:07:21.731424 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.731406 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7655e583c88021e605aee040501f02ec-config\") pod \"kube-apiserver-proxy-ip-10-0-142-24.ec2.internal\" (UID: \"7655e583c88021e605aee040501f02ec\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-24.ec2.internal" Oct 30 22:07:21.731477 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.731430 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/203b93f5f211a36c6aba561c49a0c709-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal\" (UID: \"203b93f5f211a36c6aba561c49a0c709\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal" Oct 30 22:07:21.731477 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.731447 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/203b93f5f211a36c6aba561c49a0c709-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal\" (UID: \"203b93f5f211a36c6aba561c49a0c709\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal" Oct 30 22:07:21.831581 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.831554 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/203b93f5f211a36c6aba561c49a0c709-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal\" (UID: \"203b93f5f211a36c6aba561c49a0c709\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal" Oct 30 22:07:21.831581 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.831581 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/203b93f5f211a36c6aba561c49a0c709-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal\" (UID: \"203b93f5f211a36c6aba561c49a0c709\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal" Oct 30 22:07:21.831743 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.831600 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7655e583c88021e605aee040501f02ec-config\") pod \"kube-apiserver-proxy-ip-10-0-142-24.ec2.internal\" (UID: \"7655e583c88021e605aee040501f02ec\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-24.ec2.internal" Oct 30 22:07:21.831743 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.831667 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/203b93f5f211a36c6aba561c49a0c709-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal\" (UID: \"203b93f5f211a36c6aba561c49a0c709\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal" Oct 30 22:07:21.831743 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.831717 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7655e583c88021e605aee040501f02ec-config\") pod \"kube-apiserver-proxy-ip-10-0-142-24.ec2.internal\" (UID: \"7655e583c88021e605aee040501f02ec\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-24.ec2.internal" Oct 30 22:07:21.831850 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:21.831760 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/203b93f5f211a36c6aba561c49a0c709-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal\" (UID: \"203b93f5f211a36c6aba561c49a0c709\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal" Oct 30 22:07:22.014936 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.014898 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-24.ec2.internal" Oct 30 22:07:22.017024 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.017006 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal" Oct 30 22:07:22.368468 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.368388 2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Oct 30 22:07:22.369055 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.368521 2581 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received" Oct 30 22:07:22.369055 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.368578 2581 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received" Oct 30 22:07:22.407930 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.407905 2581 apiserver.go:52] "Watching apiserver" Oct 30 22:07:22.428770 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.428750 2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Oct 30 22:07:22.452765 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.452740 2581 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Oct 30 22:07:22.453050 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.453032 2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc","openshift-dns/node-resolver-pnr6k","openshift-image-registry/node-ca-dbtlp","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal","openshift-multus/network-metrics-daemon-l7kkv","openshift-cluster-node-tuning-operator/tuned-8hv7m","openshift-multus/multus-additional-cni-plugins-v75vr","openshift-network-diagnostics/network-check-target-knzz5","kube-system/global-pull-secret-syncer-h9v2l","kube-system/konnectivity-agent-zlbzf","kube-system/kube-apiserver-proxy-ip-10-0-142-24.ec2.internal","openshift-multus/multus-f6888","openshift-network-operator/iptables-alerter-b68bn","openshift-ovn-kubernetes/ovnkube-node-ql6lh"] Oct 30 22:07:22.455678 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.455654 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.459859 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.459839 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.460788 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.460769 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dbtlp" Oct 30 22:07:22.462933 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.462916 2581 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 22:07:22.463373 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.463231 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zlbzf" Oct 30 22:07:22.464923 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.464906 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knzz5" Oct 30 22:07:22.465019 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:22.464984 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knzz5" podUID="3155dfa9-0dc2-4256-9e16-490b3721702d" Oct 30 22:07:22.467308 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.467283 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-v75vr" Oct 30 22:07:22.469655 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.469556 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7kkv" Oct 30 22:07:22.469655 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:22.469624 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l7kkv" podUID="65c2ef45-7211-4828-86bc-c9d7af39ec66" Oct 30 22:07:22.472062 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.472043 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:07:22.472157 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.472133 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Oct 30 22:07:22.472221 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:22.472203 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9v2l" podUID="cd46f3e8-c752-4eed-a65d-07e060bf32b4" Oct 30 22:07:22.472272 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.472239 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Oct 30 22:07:22.472272 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.472263 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-x4nws\"" Oct 30 22:07:22.474405 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.474370 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 22:02:21 +0000 UTC" deadline="2028-06-26 00:06:04.464220403 +0000 UTC" Oct 30 22:07:22.474405 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.474402 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14497h58m41.989820737s" Oct 30 22:07:22.474560 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.474493 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" Oct 30 22:07:22.475272 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.475245 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Oct 30 22:07:22.475445 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.475430 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-pjfxj\"" Oct 30 22:07:22.475661 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.475648 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Oct 30 22:07:22.476048 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.476035 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Oct 30 22:07:22.476209 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.476196 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Oct 30 22:07:22.477180 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.477166 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-f6888" Oct 30 22:07:22.481359 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.480508 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Oct 30 22:07:22.481359 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.481040 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-b68bn" Oct 30 22:07:22.483271 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.483252 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pnr6k" Oct 30 22:07:22.496234 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.496192 2581 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Oct 30 22:07:22.496333 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.496256 2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-7g5wc" Oct 30 22:07:22.531065 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.531046 2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Oct 30 22:07:22.535422 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.535402 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-host-var-lib-kubelet\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.535511 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.535428 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5bb239d5-4f0e-4dd6-b58b-5f3b9e014895-host-slash\") pod \"iptables-alerter-b68bn\" (UID: \"5bb239d5-4f0e-4dd6-b58b-5f3b9e014895\") " pod="openshift-network-operator/iptables-alerter-b68bn" Oct 30 22:07:22.535511 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.535453 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6b042dce-1fff-41a2-83dd-ae18f3098207-hosts-file\") pod \"node-resolver-pnr6k\" (UID: \"6b042dce-1fff-41a2-83dd-ae18f3098207\") " pod="openshift-dns/node-resolver-pnr6k" Oct 30 22:07:22.535511 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.535470 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-sys\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.535644 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.535531 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-host-slash\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.535644 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.535562 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c9381d26-e2af-4730-8b4e-06945c59585c-system-cni-dir\") pod \"multus-additional-cni-plugins-v75vr\" (UID: \"c9381d26-e2af-4730-8b4e-06945c59585c\") " pod="openshift-multus/multus-additional-cni-plugins-v75vr" Oct 30 22:07:22.535644 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.535607 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c9381d26-e2af-4730-8b4e-06945c59585c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v75vr\" (UID: \"c9381d26-e2af-4730-8b4e-06945c59585c\") " pod="openshift-multus/multus-additional-cni-plugins-v75vr" Oct 30 22:07:22.535759 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.535657 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-host\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.535759 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.535707 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-host-kubelet\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.535759 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.535727 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-node-log\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.535759 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.535742 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11464051-0aa4-4324-b2c7-7ea2096bbed6-host\") pod \"node-ca-dbtlp\" (UID: \"11464051-0aa4-4324-b2c7-7ea2096bbed6\") " pod="openshift-image-registry/node-ca-dbtlp" Oct 30 22:07:22.535887 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.535761 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f6cc017b-3f50-423a-93da-e4515336a2bc-agent-certs\") pod \"konnectivity-agent-zlbzf\" (UID: \"f6cc017b-3f50-423a-93da-e4515336a2bc\") " pod="kube-system/konnectivity-agent-zlbzf" Oct 30 22:07:22.535887 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.535822 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxdbc\" (UniqueName: \"kubernetes.io/projected/6b042dce-1fff-41a2-83dd-ae18f3098207-kube-api-access-qxdbc\") pod \"node-resolver-pnr6k\" (UID: \"6b042dce-1fff-41a2-83dd-ae18f3098207\") " pod="openshift-dns/node-resolver-pnr6k" Oct 30 22:07:22.535887 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.535881 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c9381d26-e2af-4730-8b4e-06945c59585c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v75vr\" (UID: \"c9381d26-e2af-4730-8b4e-06945c59585c\") " pod="openshift-multus/multus-additional-cni-plugins-v75vr" Oct 30 22:07:22.535977 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.535902 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpz88\" (UniqueName: \"kubernetes.io/projected/65c2ef45-7211-4828-86bc-c9d7af39ec66-kube-api-access-dpz88\") pod \"network-metrics-daemon-l7kkv\" (UID: \"65c2ef45-7211-4828-86bc-c9d7af39ec66\") " pod="openshift-multus/network-metrics-daemon-l7kkv" Oct 30 22:07:22.535977 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.535918 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2prlk\" (UniqueName: \"kubernetes.io/projected/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-kube-api-access-2prlk\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.535977 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.535943 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.535977 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.535971 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/074eeae7-932c-4642-8fea-671898135ab1-env-overrides\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.536094 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.535990 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-cnibin\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.536094 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.536019 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-etc-sysctl-conf\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.536094 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.536049 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-var-lib-kubelet\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.536094 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.536067 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-run-ovn\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.536215 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.536096 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/074eeae7-932c-4642-8fea-671898135ab1-ovnkube-script-lib\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.536215 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.536125 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xf5c4\" (UniqueName: \"kubernetes.io/projected/c9381d26-e2af-4730-8b4e-06945c59585c-kube-api-access-xf5c4\") pod \"multus-additional-cni-plugins-v75vr\" (UID: \"c9381d26-e2af-4730-8b4e-06945c59585c\") " pod="openshift-multus/multus-additional-cni-plugins-v75vr" Oct 30 22:07:22.536215 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.536155 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-host-run-netns\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.536215 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.536189 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-multus-conf-dir\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.536358 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.536249 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrwhs\" (UniqueName: \"kubernetes.io/projected/11464051-0aa4-4324-b2c7-7ea2096bbed6-kube-api-access-nrwhs\") pod \"node-ca-dbtlp\" (UID: \"11464051-0aa4-4324-b2c7-7ea2096bbed6\") " pod="openshift-image-registry/node-ca-dbtlp" Oct 30 22:07:22.536358 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.536278 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-host-run-netns\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.536358 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.536303 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-etc-openvswitch\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.536358 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.536323 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-host-run-multus-certs\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.536358 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.536353 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-etc-kubernetes\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.536523 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.536371 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-etc-modprobe-d\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.536523 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.536393 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-var-lib-openvswitch\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.536523 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.536420 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c9381d26-e2af-4730-8b4e-06945c59585c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v75vr\" (UID: \"c9381d26-e2af-4730-8b4e-06945c59585c\") " pod="openshift-multus/multus-additional-cni-plugins-v75vr" Oct 30 22:07:22.536523 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.536452 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cd46f3e8-c752-4eed-a65d-07e060bf32b4-kubelet-config\") pod \"global-pull-secret-syncer-h9v2l\" (UID: \"cd46f3e8-c752-4eed-a65d-07e060bf32b4\") " pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:07:22.536523 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.536471 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-system-cni-dir\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.536523 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.536511 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-cni-binary-copy\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.536784 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.536561 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-host-var-lib-cni-multus\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.536784 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.536594 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knh4d\" (UniqueName: \"kubernetes.io/projected/5bb239d5-4f0e-4dd6-b58b-5f3b9e014895-kube-api-access-knh4d\") pod \"iptables-alerter-b68bn\" (UID: \"5bb239d5-4f0e-4dd6-b58b-5f3b9e014895\") " pod="openshift-network-operator/iptables-alerter-b68bn" Oct 30 22:07:22.536784 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.536635 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-etc-sysctl-d\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.536784 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.536672 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-lib-modules\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.536784 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.536694 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-host-run-ovn-kubernetes\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.536784 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.536717 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e5779bc8-21ad-465e-8749-8f7e9afdc03f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c69jc\" (UID: \"e5779bc8-21ad-465e-8749-8f7e9afdc03f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" Oct 30 22:07:22.536784 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.536743 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7f59\" (UniqueName: \"kubernetes.io/projected/e5779bc8-21ad-465e-8749-8f7e9afdc03f-kube-api-access-h7f59\") pod \"aws-ebs-csi-driver-node-c69jc\" (UID: \"e5779bc8-21ad-465e-8749-8f7e9afdc03f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" Oct 30 22:07:22.536784 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.536784 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-multus-daemon-config\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.537099 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.536811 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b042dce-1fff-41a2-83dd-ae18f3098207-tmp-dir\") pod \"node-resolver-pnr6k\" (UID: \"6b042dce-1fff-41a2-83dd-ae18f3098207\") " pod="openshift-dns/node-resolver-pnr6k" Oct 30 22:07:22.537099 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.536838 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-etc-systemd\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.537099 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.536883 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cd46f3e8-c752-4eed-a65d-07e060bf32b4-original-pull-secret\") pod \"global-pull-secret-syncer-h9v2l\" (UID: \"cd46f3e8-c752-4eed-a65d-07e060bf32b4\") " pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:07:22.537099 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.536911 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e5779bc8-21ad-465e-8749-8f7e9afdc03f-registration-dir\") pod \"aws-ebs-csi-driver-node-c69jc\" (UID: \"e5779bc8-21ad-465e-8749-8f7e9afdc03f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" Oct 30 22:07:22.537099 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.536937 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-hostroot\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.537099 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.536959 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/11464051-0aa4-4324-b2c7-7ea2096bbed6-serviceca\") pod \"node-ca-dbtlp\" (UID: \"11464051-0aa4-4324-b2c7-7ea2096bbed6\") " pod="openshift-image-registry/node-ca-dbtlp" Oct 30 22:07:22.537099 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.536983 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-tmp\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.537099 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.537014 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-host-cni-bin\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.537099 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.537033 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cd46f3e8-c752-4eed-a65d-07e060bf32b4-dbus\") pod \"global-pull-secret-syncer-h9v2l\" (UID: \"cd46f3e8-c752-4eed-a65d-07e060bf32b4\") " pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:07:22.537099 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.537055 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-multus-socket-dir-parent\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.537099 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.537079 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5bb239d5-4f0e-4dd6-b58b-5f3b9e014895-iptables-alerter-script\") pod \"iptables-alerter-b68bn\" (UID: \"5bb239d5-4f0e-4dd6-b58b-5f3b9e014895\") " pod="openshift-network-operator/iptables-alerter-b68bn" Oct 30 22:07:22.537099 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.537100 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f6cc017b-3f50-423a-93da-e4515336a2bc-konnectivity-ca\") pod \"konnectivity-agent-zlbzf\" (UID: \"f6cc017b-3f50-423a-93da-e4515336a2bc\") " pod="kube-system/konnectivity-agent-zlbzf" Oct 30 22:07:22.537480 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.537135 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-etc-sysconfig\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.537480 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.537152 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-run\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.537480 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.537195 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/074eeae7-932c-4642-8fea-671898135ab1-ovn-node-metrics-cert\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.537480 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.537220 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c9381d26-e2af-4730-8b4e-06945c59585c-cnibin\") pod \"multus-additional-cni-plugins-v75vr\" (UID: \"c9381d26-e2af-4730-8b4e-06945c59585c\") " pod="openshift-multus/multus-additional-cni-plugins-v75vr" Oct 30 22:07:22.537480 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.537248 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-host-var-lib-cni-bin\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.537480 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.537337 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65c2ef45-7211-4828-86bc-c9d7af39ec66-metrics-certs\") pod \"network-metrics-daemon-l7kkv\" (UID: \"65c2ef45-7211-4828-86bc-c9d7af39ec66\") " pod="openshift-multus/network-metrics-daemon-l7kkv" Oct 30 22:07:22.537480 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.537357 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-systemd-units\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.537480 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.537378 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-run-systemd\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.537480 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.537402 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-host-cni-netd\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.537480 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.537417 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/074eeae7-932c-4642-8fea-671898135ab1-ovnkube-config\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.537480 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.537440 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzvcp\" (UniqueName: \"kubernetes.io/projected/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-kube-api-access-hzvcp\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.537480 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.537456 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-etc-tuned\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.537480 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.537470 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-log-socket\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.537851 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.537499 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gvkg\" (UniqueName: \"kubernetes.io/projected/3155dfa9-0dc2-4256-9e16-490b3721702d-kube-api-access-2gvkg\") pod \"network-check-target-knzz5\" (UID: \"3155dfa9-0dc2-4256-9e16-490b3721702d\") " pod="openshift-network-diagnostics/network-check-target-knzz5" Oct 30 22:07:22.537851 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.537533 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c9381d26-e2af-4730-8b4e-06945c59585c-cni-binary-copy\") pod \"multus-additional-cni-plugins-v75vr\" (UID: \"c9381d26-e2af-4730-8b4e-06945c59585c\") " pod="openshift-multus/multus-additional-cni-plugins-v75vr" Oct 30 22:07:22.537851 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.537562 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e5779bc8-21ad-465e-8749-8f7e9afdc03f-socket-dir\") pod \"aws-ebs-csi-driver-node-c69jc\" (UID: \"e5779bc8-21ad-465e-8749-8f7e9afdc03f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" Oct 30 22:07:22.537851 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.537586 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-host-run-k8s-cni-cncf-io\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.537851 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.537602 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-run-openvswitch\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.537851 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.537631 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e5779bc8-21ad-465e-8749-8f7e9afdc03f-device-dir\") pod \"aws-ebs-csi-driver-node-c69jc\" (UID: \"e5779bc8-21ad-465e-8749-8f7e9afdc03f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" Oct 30 22:07:22.537851 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.537654 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e5779bc8-21ad-465e-8749-8f7e9afdc03f-etc-selinux\") pod \"aws-ebs-csi-driver-node-c69jc\" (UID: \"e5779bc8-21ad-465e-8749-8f7e9afdc03f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" Oct 30 22:07:22.537851 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.537682 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-multus-cni-dir\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.537851 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.537699 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-etc-kubernetes\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.537851 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.537717 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kxhz\" (UniqueName: \"kubernetes.io/projected/074eeae7-932c-4642-8fea-671898135ab1-kube-api-access-8kxhz\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.537851 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.537739 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e5779bc8-21ad-465e-8749-8f7e9afdc03f-sys-fs\") pod \"aws-ebs-csi-driver-node-c69jc\" (UID: \"e5779bc8-21ad-465e-8749-8f7e9afdc03f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" Oct 30 22:07:22.581272 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.581221 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Oct 30 22:07:22.581479 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.581352 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Oct 30 22:07:22.581592 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.581570 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Oct 30 22:07:22.581592 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.581577 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Oct 30 22:07:22.581799 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.581780 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Oct 30 22:07:22.581865 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.581788 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Oct 30 22:07:22.581922 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.581864 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Oct 30 22:07:22.581922 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.581874 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Oct 30 22:07:22.581922 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.581913 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Oct 30 22:07:22.582056 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.581935 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Oct 30 22:07:22.582056 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.581940 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Oct 30 22:07:22.582056 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.581954 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Oct 30 22:07:22.582056 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.581970 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Oct 30 22:07:22.582056 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.581981 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Oct 30 22:07:22.582056 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.581988 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-7l7rd\"" Oct 30 22:07:22.582056 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.582000 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Oct 30 22:07:22.582056 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.582017 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Oct 30 22:07:22.582438 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.582081 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Oct 30 22:07:22.584196 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.584096 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-vdq5j\"" Oct 30 22:07:22.584196 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.584175 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-s9t4n\"" Oct 30 22:07:22.584196 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.584183 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gfzqs\"" Oct 30 22:07:22.584196 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.584176 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-22mqf\"" Oct 30 22:07:22.584424 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.584177 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Oct 30 22:07:22.584424 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.584205 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Oct 30 22:07:22.584424 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.584215 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Oct 30 22:07:22.584424 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.584239 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-dzfp9\"" Oct 30 22:07:22.584424 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.584249 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zgfdn\"" Oct 30 22:07:22.638954 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.638888 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5bb239d5-4f0e-4dd6-b58b-5f3b9e014895-host-slash\") pod \"iptables-alerter-b68bn\" (UID: \"5bb239d5-4f0e-4dd6-b58b-5f3b9e014895\") " pod="openshift-network-operator/iptables-alerter-b68bn" Oct 30 22:07:22.638954 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.638924 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6b042dce-1fff-41a2-83dd-ae18f3098207-hosts-file\") pod \"node-resolver-pnr6k\" (UID: \"6b042dce-1fff-41a2-83dd-ae18f3098207\") " pod="openshift-dns/node-resolver-pnr6k" Oct 30 22:07:22.638954 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.638942 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-sys\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.639245 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.638963 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-host-slash\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.639367 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.639339 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6b042dce-1fff-41a2-83dd-ae18f3098207-hosts-file\") pod \"node-resolver-pnr6k\" (UID: \"6b042dce-1fff-41a2-83dd-ae18f3098207\") " pod="openshift-dns/node-resolver-pnr6k" Oct 30 22:07:22.639429 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.639345 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c9381d26-e2af-4730-8b4e-06945c59585c-system-cni-dir\") pod \"multus-additional-cni-plugins-v75vr\" (UID: \"c9381d26-e2af-4730-8b4e-06945c59585c\") " pod="openshift-multus/multus-additional-cni-plugins-v75vr" Oct 30 22:07:22.639429 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.639378 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-sys\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.639429 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.639393 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-host-slash\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.639528 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.639427 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c9381d26-e2af-4730-8b4e-06945c59585c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v75vr\" (UID: \"c9381d26-e2af-4730-8b4e-06945c59585c\") " pod="openshift-multus/multus-additional-cni-plugins-v75vr" Oct 30 22:07:22.639528 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.639086 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c9381d26-e2af-4730-8b4e-06945c59585c-system-cni-dir\") pod \"multus-additional-cni-plugins-v75vr\" (UID: \"c9381d26-e2af-4730-8b4e-06945c59585c\") " pod="openshift-multus/multus-additional-cni-plugins-v75vr" Oct 30 22:07:22.639528 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.639478 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-host\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.639528 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.639475 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5bb239d5-4f0e-4dd6-b58b-5f3b9e014895-host-slash\") pod \"iptables-alerter-b68bn\" (UID: \"5bb239d5-4f0e-4dd6-b58b-5f3b9e014895\") " pod="openshift-network-operator/iptables-alerter-b68bn" Oct 30 22:07:22.639528 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.639514 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-host-kubelet\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.639719 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.639547 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-node-log\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.639719 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.639579 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11464051-0aa4-4324-b2c7-7ea2096bbed6-host\") pod \"node-ca-dbtlp\" (UID: \"11464051-0aa4-4324-b2c7-7ea2096bbed6\") " pod="openshift-image-registry/node-ca-dbtlp" Oct 30 22:07:22.639719 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.639575 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-host\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.639719 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.639606 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-host-kubelet\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.639719 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.639633 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-node-log\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.639719 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.639664 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11464051-0aa4-4324-b2c7-7ea2096bbed6-host\") pod \"node-ca-dbtlp\" (UID: \"11464051-0aa4-4324-b2c7-7ea2096bbed6\") " pod="openshift-image-registry/node-ca-dbtlp" Oct 30 22:07:22.639719 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.639665 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f6cc017b-3f50-423a-93da-e4515336a2bc-agent-certs\") pod \"konnectivity-agent-zlbzf\" (UID: \"f6cc017b-3f50-423a-93da-e4515336a2bc\") " pod="kube-system/konnectivity-agent-zlbzf" Oct 30 22:07:22.639719 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.639692 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxdbc\" (UniqueName: \"kubernetes.io/projected/6b042dce-1fff-41a2-83dd-ae18f3098207-kube-api-access-qxdbc\") pod \"node-resolver-pnr6k\" (UID: \"6b042dce-1fff-41a2-83dd-ae18f3098207\") " pod="openshift-dns/node-resolver-pnr6k" Oct 30 22:07:22.639719 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.639720 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c9381d26-e2af-4730-8b4e-06945c59585c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v75vr\" (UID: \"c9381d26-e2af-4730-8b4e-06945c59585c\") " pod="openshift-multus/multus-additional-cni-plugins-v75vr" Oct 30 22:07:22.640213 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.639755 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpz88\" (UniqueName: \"kubernetes.io/projected/65c2ef45-7211-4828-86bc-c9d7af39ec66-kube-api-access-dpz88\") pod \"network-metrics-daemon-l7kkv\" (UID: \"65c2ef45-7211-4828-86bc-c9d7af39ec66\") " pod="openshift-multus/network-metrics-daemon-l7kkv" Oct 30 22:07:22.640213 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.639788 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2prlk\" (UniqueName: \"kubernetes.io/projected/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-kube-api-access-2prlk\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.640213 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.639816 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.640213 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.639850 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/074eeae7-932c-4642-8fea-671898135ab1-env-overrides\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.640213 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.639883 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-cnibin\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.640213 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.639937 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c9381d26-e2af-4730-8b4e-06945c59585c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v75vr\" (UID: \"c9381d26-e2af-4730-8b4e-06945c59585c\") " pod="openshift-multus/multus-additional-cni-plugins-v75vr" Oct 30 22:07:22.640213 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.639959 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-cnibin\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.640213 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.640101 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-etc-sysctl-conf\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.640213 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.640121 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c9381d26-e2af-4730-8b4e-06945c59585c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v75vr\" (UID: \"c9381d26-e2af-4730-8b4e-06945c59585c\") " pod="openshift-multus/multus-additional-cni-plugins-v75vr" Oct 30 22:07:22.640213 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.640127 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-var-lib-kubelet\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.640213 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.640187 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-run-ovn\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.640213 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.640200 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.640213 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.640125 2581 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Oct 30 22:07:22.640898 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.640224 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/074eeae7-932c-4642-8fea-671898135ab1-ovnkube-script-lib\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.640898 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.640289 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-run-ovn\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.640898 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.640300 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf5c4\" (UniqueName: \"kubernetes.io/projected/c9381d26-e2af-4730-8b4e-06945c59585c-kube-api-access-xf5c4\") pod \"multus-additional-cni-plugins-v75vr\" (UID: \"c9381d26-e2af-4730-8b4e-06945c59585c\") " pod="openshift-multus/multus-additional-cni-plugins-v75vr" Oct 30 22:07:22.640898 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.640315 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-var-lib-kubelet\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.640898 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.640329 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-host-run-netns\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.640898 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.640361 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-multus-conf-dir\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.640898 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.640434 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrwhs\" (UniqueName: \"kubernetes.io/projected/11464051-0aa4-4324-b2c7-7ea2096bbed6-kube-api-access-nrwhs\") pod \"node-ca-dbtlp\" (UID: \"11464051-0aa4-4324-b2c7-7ea2096bbed6\") " pod="openshift-image-registry/node-ca-dbtlp" Oct 30 22:07:22.640898 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.640578 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-host-run-netns\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.640898 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.640623 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-etc-openvswitch\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.640898 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.640652 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-host-run-multus-certs\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.640898 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.640656 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-etc-sysctl-conf\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.640898 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.640685 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-etc-kubernetes\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.640898 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.640721 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-etc-modprobe-d\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.640898 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.640793 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-host-run-netns\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.640898 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.640843 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-var-lib-openvswitch\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.640898 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.640888 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c9381d26-e2af-4730-8b4e-06945c59585c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v75vr\" (UID: \"c9381d26-e2af-4730-8b4e-06945c59585c\") " pod="openshift-multus/multus-additional-cni-plugins-v75vr" Oct 30 22:07:22.641711 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.641009 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-etc-modprobe-d\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.641711 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.641066 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/074eeae7-932c-4642-8fea-671898135ab1-ovnkube-script-lib\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.641711 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.641071 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/074eeae7-932c-4642-8fea-671898135ab1-env-overrides\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.641711 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.641124 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-host-run-netns\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.641711 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.641137 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-var-lib-openvswitch\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.641711 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.641195 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-host-run-multus-certs\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.641711 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.641210 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-etc-kubernetes\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.641711 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.641217 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cd46f3e8-c752-4eed-a65d-07e060bf32b4-kubelet-config\") pod \"global-pull-secret-syncer-h9v2l\" (UID: \"cd46f3e8-c752-4eed-a65d-07e060bf32b4\") " pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:07:22.641711 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.641263 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-system-cni-dir\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.641711 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.641321 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-cni-binary-copy\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.641711 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.641356 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-system-cni-dir\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.641711 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.641358 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-host-var-lib-cni-multus\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.641711 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.641412 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knh4d\" (UniqueName: \"kubernetes.io/projected/5bb239d5-4f0e-4dd6-b58b-5f3b9e014895-kube-api-access-knh4d\") pod \"iptables-alerter-b68bn\" (UID: \"5bb239d5-4f0e-4dd6-b58b-5f3b9e014895\") " pod="openshift-network-operator/iptables-alerter-b68bn" Oct 30 22:07:22.641711 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.641448 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-etc-sysctl-d\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.641711 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.641483 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-lib-modules\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.641711 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.641262 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-etc-openvswitch\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.641711 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.641484 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-host-var-lib-cni-multus\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.641711 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.641555 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cd46f3e8-c752-4eed-a65d-07e060bf32b4-kubelet-config\") pod \"global-pull-secret-syncer-h9v2l\" (UID: \"cd46f3e8-c752-4eed-a65d-07e060bf32b4\") " pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:07:22.642367 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.641623 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-etc-sysctl-d\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.642367 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.641667 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-host-run-ovn-kubernetes\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.642367 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.641265 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-multus-conf-dir\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.642367 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.641701 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-lib-modules\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.642367 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.641714 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e5779bc8-21ad-465e-8749-8f7e9afdc03f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c69jc\" (UID: \"e5779bc8-21ad-465e-8749-8f7e9afdc03f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" Oct 30 22:07:22.642367 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.641739 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-host-run-ovn-kubernetes\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.642367 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.641764 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7f59\" (UniqueName: \"kubernetes.io/projected/e5779bc8-21ad-465e-8749-8f7e9afdc03f-kube-api-access-h7f59\") pod \"aws-ebs-csi-driver-node-c69jc\" (UID: \"e5779bc8-21ad-465e-8749-8f7e9afdc03f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" Oct 30 22:07:22.642367 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.641771 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e5779bc8-21ad-465e-8749-8f7e9afdc03f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c69jc\" (UID: \"e5779bc8-21ad-465e-8749-8f7e9afdc03f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" Oct 30 22:07:22.642367 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.641794 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-multus-daemon-config\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.642367 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.641828 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b042dce-1fff-41a2-83dd-ae18f3098207-tmp-dir\") pod \"node-resolver-pnr6k\" (UID: \"6b042dce-1fff-41a2-83dd-ae18f3098207\") " pod="openshift-dns/node-resolver-pnr6k" Oct 30 22:07:22.642367 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.641857 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-etc-systemd\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.642367 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.641889 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cd46f3e8-c752-4eed-a65d-07e060bf32b4-original-pull-secret\") pod \"global-pull-secret-syncer-h9v2l\" (UID: \"cd46f3e8-c752-4eed-a65d-07e060bf32b4\") " pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:07:22.642367 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.641915 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e5779bc8-21ad-465e-8749-8f7e9afdc03f-registration-dir\") pod \"aws-ebs-csi-driver-node-c69jc\" (UID: \"e5779bc8-21ad-465e-8749-8f7e9afdc03f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" Oct 30 22:07:22.642367 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.641945 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-hostroot\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.642367 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.641973 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/11464051-0aa4-4324-b2c7-7ea2096bbed6-serviceca\") pod \"node-ca-dbtlp\" (UID: \"11464051-0aa4-4324-b2c7-7ea2096bbed6\") " pod="openshift-image-registry/node-ca-dbtlp" Oct 30 22:07:22.642367 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.642009 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-tmp\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.642367 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:22.642046 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 22:07:22.642367 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.642038 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-host-cni-bin\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.643130 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.642176 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cd46f3e8-c752-4eed-a65d-07e060bf32b4-dbus\") pod \"global-pull-secret-syncer-h9v2l\" (UID: \"cd46f3e8-c752-4eed-a65d-07e060bf32b4\") " pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:07:22.643130 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.642201 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-multus-socket-dir-parent\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.643130 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:22.642244 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd46f3e8-c752-4eed-a65d-07e060bf32b4-original-pull-secret podName:cd46f3e8-c752-4eed-a65d-07e060bf32b4 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:23.142199281 +0000 UTC m=+2.068301599 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cd46f3e8-c752-4eed-a65d-07e060bf32b4-original-pull-secret") pod "global-pull-secret-syncer-h9v2l" (UID: "cd46f3e8-c752-4eed-a65d-07e060bf32b4") : object "kube-system"/"original-pull-secret" not registered Oct 30 22:07:22.643130 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.642303 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5bb239d5-4f0e-4dd6-b58b-5f3b9e014895-iptables-alerter-script\") pod \"iptables-alerter-b68bn\" (UID: \"5bb239d5-4f0e-4dd6-b58b-5f3b9e014895\") " pod="openshift-network-operator/iptables-alerter-b68bn" Oct 30 22:07:22.643130 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.642345 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f6cc017b-3f50-423a-93da-e4515336a2bc-konnectivity-ca\") pod \"konnectivity-agent-zlbzf\" (UID: \"f6cc017b-3f50-423a-93da-e4515336a2bc\") " pod="kube-system/konnectivity-agent-zlbzf" Oct 30 22:07:22.643130 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.642382 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-etc-sysconfig\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.643130 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.642386 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e5779bc8-21ad-465e-8749-8f7e9afdc03f-registration-dir\") pod \"aws-ebs-csi-driver-node-c69jc\" (UID: \"e5779bc8-21ad-465e-8749-8f7e9afdc03f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" Oct 30 22:07:22.643130 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.642416 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-run\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.643130 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.642436 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/074eeae7-932c-4642-8fea-671898135ab1-ovn-node-metrics-cert\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.643130 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.642449 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-hostroot\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.643130 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.642455 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c9381d26-e2af-4730-8b4e-06945c59585c-cnibin\") pod \"multus-additional-cni-plugins-v75vr\" (UID: \"c9381d26-e2af-4730-8b4e-06945c59585c\") " pod="openshift-multus/multus-additional-cni-plugins-v75vr" Oct 30 22:07:22.643130 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.642477 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-host-var-lib-cni-bin\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.643130 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.642498 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65c2ef45-7211-4828-86bc-c9d7af39ec66-metrics-certs\") pod \"network-metrics-daemon-l7kkv\" (UID: \"65c2ef45-7211-4828-86bc-c9d7af39ec66\") " pod="openshift-multus/network-metrics-daemon-l7kkv" Oct 30 22:07:22.643130 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.642517 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-systemd-units\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.643130 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.642534 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-run-systemd\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.643130 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.642554 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-host-cni-netd\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.644131 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.642572 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/074eeae7-932c-4642-8fea-671898135ab1-ovnkube-config\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.644131 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.642592 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzvcp\" (UniqueName: \"kubernetes.io/projected/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-kube-api-access-hzvcp\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.644131 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.642610 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-etc-tuned\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.644131 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.642629 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-log-socket\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.644131 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.642654 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gvkg\" (UniqueName: \"kubernetes.io/projected/3155dfa9-0dc2-4256-9e16-490b3721702d-kube-api-access-2gvkg\") pod \"network-check-target-knzz5\" (UID: \"3155dfa9-0dc2-4256-9e16-490b3721702d\") " pod="openshift-network-diagnostics/network-check-target-knzz5" Oct 30 22:07:22.644131 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.642675 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c9381d26-e2af-4730-8b4e-06945c59585c-cni-binary-copy\") pod \"multus-additional-cni-plugins-v75vr\" (UID: \"c9381d26-e2af-4730-8b4e-06945c59585c\") " pod="openshift-multus/multus-additional-cni-plugins-v75vr" Oct 30 22:07:22.644131 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.642703 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e5779bc8-21ad-465e-8749-8f7e9afdc03f-socket-dir\") pod \"aws-ebs-csi-driver-node-c69jc\" (UID: \"e5779bc8-21ad-465e-8749-8f7e9afdc03f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" Oct 30 22:07:22.644131 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.642711 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-host-cni-bin\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.644131 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.642735 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-host-run-k8s-cni-cncf-io\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.644131 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.642771 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-run-openvswitch\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.644131 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.642811 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e5779bc8-21ad-465e-8749-8f7e9afdc03f-device-dir\") pod \"aws-ebs-csi-driver-node-c69jc\" (UID: \"e5779bc8-21ad-465e-8749-8f7e9afdc03f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" Oct 30 22:07:22.644131 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.642885 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e5779bc8-21ad-465e-8749-8f7e9afdc03f-device-dir\") pod \"aws-ebs-csi-driver-node-c69jc\" (UID: \"e5779bc8-21ad-465e-8749-8f7e9afdc03f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" Oct 30 22:07:22.644131 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.642891 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-systemd-units\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.644131 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.642953 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-host-cni-netd\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.644131 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.642940 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e5779bc8-21ad-465e-8749-8f7e9afdc03f-etc-selinux\") pod \"aws-ebs-csi-driver-node-c69jc\" (UID: \"e5779bc8-21ad-465e-8749-8f7e9afdc03f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" Oct 30 22:07:22.644131 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.643005 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-multus-cni-dir\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.644131 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.643042 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-etc-kubernetes\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.645088 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.643044 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e5779bc8-21ad-465e-8749-8f7e9afdc03f-etc-selinux\") pod \"aws-ebs-csi-driver-node-c69jc\" (UID: \"e5779bc8-21ad-465e-8749-8f7e9afdc03f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" Oct 30 22:07:22.645088 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.643062 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-cni-binary-copy\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.645088 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.643147 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-etc-systemd\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.645088 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.643209 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-etc-kubernetes\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.645088 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.643601 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/074eeae7-932c-4642-8fea-671898135ab1-ovnkube-config\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.645088 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.643649 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kxhz\" (UniqueName: \"kubernetes.io/projected/074eeae7-932c-4642-8fea-671898135ab1-kube-api-access-8kxhz\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.645088 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.643688 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e5779bc8-21ad-465e-8749-8f7e9afdc03f-sys-fs\") pod \"aws-ebs-csi-driver-node-c69jc\" (UID: \"e5779bc8-21ad-465e-8749-8f7e9afdc03f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" Oct 30 22:07:22.645088 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.643723 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-host-var-lib-kubelet\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.645088 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.643728 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c9381d26-e2af-4730-8b4e-06945c59585c-cni-binary-copy\") pod \"multus-additional-cni-plugins-v75vr\" (UID: \"c9381d26-e2af-4730-8b4e-06945c59585c\") " pod="openshift-multus/multus-additional-cni-plugins-v75vr" Oct 30 22:07:22.645088 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.644164 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5bb239d5-4f0e-4dd6-b58b-5f3b9e014895-iptables-alerter-script\") pod \"iptables-alerter-b68bn\" (UID: \"5bb239d5-4f0e-4dd6-b58b-5f3b9e014895\") " pod="openshift-network-operator/iptables-alerter-b68bn" Oct 30 22:07:22.645088 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.644271 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c9381d26-e2af-4730-8b4e-06945c59585c-cnibin\") pod \"multus-additional-cni-plugins-v75vr\" (UID: \"c9381d26-e2af-4730-8b4e-06945c59585c\") " pod="openshift-multus/multus-additional-cni-plugins-v75vr" Oct 30 22:07:22.645088 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.644329 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-host-var-lib-cni-bin\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.645088 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:22.644422 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 22:07:22.645088 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:22.644486 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65c2ef45-7211-4828-86bc-c9d7af39ec66-metrics-certs podName:65c2ef45-7211-4828-86bc-c9d7af39ec66 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:23.144469169 +0000 UTC m=+2.070571475 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/65c2ef45-7211-4828-86bc-c9d7af39ec66-metrics-certs") pod "network-metrics-daemon-l7kkv" (UID: "65c2ef45-7211-4828-86bc-c9d7af39ec66") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 22:07:22.645088 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.644528 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-run-systemd\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.645088 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.644574 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-log-socket\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.645088 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.644717 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b042dce-1fff-41a2-83dd-ae18f3098207-tmp-dir\") pod \"node-resolver-pnr6k\" (UID: \"6b042dce-1fff-41a2-83dd-ae18f3098207\") " pod="openshift-dns/node-resolver-pnr6k" Oct 30 22:07:22.646659 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.645078 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-multus-daemon-config\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.646851 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.645348 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c9381d26-e2af-4730-8b4e-06945c59585c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v75vr\" (UID: \"c9381d26-e2af-4730-8b4e-06945c59585c\") " pod="openshift-multus/multus-additional-cni-plugins-v75vr" Oct 30 22:07:22.646969 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.645364 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-multus-socket-dir-parent\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.646969 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.645426 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-etc-sysconfig\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.646969 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.645475 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-run\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.646969 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.645495 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cd46f3e8-c752-4eed-a65d-07e060bf32b4-dbus\") pod \"global-pull-secret-syncer-h9v2l\" (UID: \"cd46f3e8-c752-4eed-a65d-07e060bf32b4\") " pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:07:22.646969 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.646898 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/074eeae7-932c-4642-8fea-671898135ab1-ovn-node-metrics-cert\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.646969 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.645982 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f6cc017b-3f50-423a-93da-e4515336a2bc-agent-certs\") pod \"konnectivity-agent-zlbzf\" (UID: \"f6cc017b-3f50-423a-93da-e4515336a2bc\") " pod="kube-system/konnectivity-agent-zlbzf" Oct 30 22:07:22.646969 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.646217 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-host-run-k8s-cni-cncf-io\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.646969 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.646259 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-run-openvswitch\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.646969 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.646263 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-tmp\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.646969 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.646265 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-multus-cni-dir\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.647524 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.646039 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f6cc017b-3f50-423a-93da-e4515336a2bc-konnectivity-ca\") pod \"konnectivity-agent-zlbzf\" (UID: \"f6cc017b-3f50-423a-93da-e4515336a2bc\") " pod="kube-system/konnectivity-agent-zlbzf" Oct 30 22:07:22.647524 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.646332 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e5779bc8-21ad-465e-8749-8f7e9afdc03f-sys-fs\") pod \"aws-ebs-csi-driver-node-c69jc\" (UID: \"e5779bc8-21ad-465e-8749-8f7e9afdc03f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" Oct 30 22:07:22.647524 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.647000 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/11464051-0aa4-4324-b2c7-7ea2096bbed6-serviceca\") pod \"node-ca-dbtlp\" (UID: \"11464051-0aa4-4324-b2c7-7ea2096bbed6\") " pod="openshift-image-registry/node-ca-dbtlp" Oct 30 22:07:22.647524 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.646173 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e5779bc8-21ad-465e-8749-8f7e9afdc03f-socket-dir\") pod \"aws-ebs-csi-driver-node-c69jc\" (UID: \"e5779bc8-21ad-465e-8749-8f7e9afdc03f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" Oct 30 22:07:22.647524 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.646388 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-host-var-lib-kubelet\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.648522 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.648503 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-etc-tuned\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.666813 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.666779 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrwhs\" (UniqueName: \"kubernetes.io/projected/11464051-0aa4-4324-b2c7-7ea2096bbed6-kube-api-access-nrwhs\") pod \"node-ca-dbtlp\" (UID: \"11464051-0aa4-4324-b2c7-7ea2096bbed6\") " pod="openshift-image-registry/node-ca-dbtlp" Oct 30 22:07:22.667535 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.667515 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7f59\" (UniqueName: \"kubernetes.io/projected/e5779bc8-21ad-465e-8749-8f7e9afdc03f-kube-api-access-h7f59\") pod \"aws-ebs-csi-driver-node-c69jc\" (UID: \"e5779bc8-21ad-465e-8749-8f7e9afdc03f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" Oct 30 22:07:22.667619 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.667585 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2prlk\" (UniqueName: \"kubernetes.io/projected/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-kube-api-access-2prlk\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.667619 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.667605 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf5c4\" (UniqueName: \"kubernetes.io/projected/c9381d26-e2af-4730-8b4e-06945c59585c-kube-api-access-xf5c4\") pod \"multus-additional-cni-plugins-v75vr\" (UID: \"c9381d26-e2af-4730-8b4e-06945c59585c\") " pod="openshift-multus/multus-additional-cni-plugins-v75vr" Oct 30 22:07:22.667966 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.667945 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-knh4d\" (UniqueName: \"kubernetes.io/projected/5bb239d5-4f0e-4dd6-b58b-5f3b9e014895-kube-api-access-knh4d\") pod \"iptables-alerter-b68bn\" (UID: \"5bb239d5-4f0e-4dd6-b58b-5f3b9e014895\") " pod="openshift-network-operator/iptables-alerter-b68bn" Oct 30 22:07:22.668074 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.668051 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxdbc\" (UniqueName: \"kubernetes.io/projected/6b042dce-1fff-41a2-83dd-ae18f3098207-kube-api-access-qxdbc\") pod \"node-resolver-pnr6k\" (UID: \"6b042dce-1fff-41a2-83dd-ae18f3098207\") " pod="openshift-dns/node-resolver-pnr6k" Oct 30 22:07:22.668960 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.668938 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpz88\" (UniqueName: \"kubernetes.io/projected/65c2ef45-7211-4828-86bc-c9d7af39ec66-kube-api-access-dpz88\") pod \"network-metrics-daemon-l7kkv\" (UID: \"65c2ef45-7211-4828-86bc-c9d7af39ec66\") " pod="openshift-multus/network-metrics-daemon-l7kkv" Oct 30 22:07:22.669411 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:22.669392 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 22:07:22.669516 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:22.669423 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 22:07:22.669516 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:22.669437 2581 projected.go:196] Error preparing data for projected volume kube-api-access-2gvkg for pod openshift-network-diagnostics/network-check-target-knzz5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 22:07:22.669516 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:22.669497 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3155dfa9-0dc2-4256-9e16-490b3721702d-kube-api-access-2gvkg podName:3155dfa9-0dc2-4256-9e16-490b3721702d nodeName:}" failed. No retries permitted until 2026-10-30 22:07:23.169479919 +0000 UTC m=+2.095582247 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2gvkg" (UniqueName: "kubernetes.io/projected/3155dfa9-0dc2-4256-9e16-490b3721702d-kube-api-access-2gvkg") pod "network-check-target-knzz5" (UID: "3155dfa9-0dc2-4256-9e16-490b3721702d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 22:07:22.671058 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.671039 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzvcp\" (UniqueName: \"kubernetes.io/projected/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-kube-api-access-hzvcp\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:07:22.671890 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.671868 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kxhz\" (UniqueName: \"kubernetes.io/projected/074eeae7-932c-4642-8fea-671898135ab1-kube-api-access-8kxhz\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.721793 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:22.721758 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod203b93f5f211a36c6aba561c49a0c709.slice/crio-61a90835359d781127b7c1b0c8148cf6aef8e3c7ad74148743763ac42eb9b7ad WatchSource:0}: Error finding container 61a90835359d781127b7c1b0c8148cf6aef8e3c7ad74148743763ac42eb9b7ad: Status 404 returned error can't find the container with id 61a90835359d781127b7c1b0c8148cf6aef8e3c7ad74148743763ac42eb9b7ad Oct 30 22:07:22.722411 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:22.722386 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7655e583c88021e605aee040501f02ec.slice/crio-36051ba353f4d1493e47afd3bee1fa39336698eac55c1ddd7a97c40dbd52bdfa WatchSource:0}: Error finding container 36051ba353f4d1493e47afd3bee1fa39336698eac55c1ddd7a97c40dbd52bdfa: Status 404 returned error can't find the container with id 36051ba353f4d1493e47afd3bee1fa39336698eac55c1ddd7a97c40dbd52bdfa Oct 30 22:07:22.728161 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.728140 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 22:07:22.787801 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.787777 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:07:22.798225 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:22.798199 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode74d4847_dcbb_4b6a_9a4e_dc3a49ef7a44.slice/crio-98868e04e6322ac36774e72846ecdcde8d21ab195d25357a997c6dcead2ba090 WatchSource:0}: Error finding container 98868e04e6322ac36774e72846ecdcde8d21ab195d25357a997c6dcead2ba090: Status 404 returned error can't find the container with id 98868e04e6322ac36774e72846ecdcde8d21ab195d25357a997c6dcead2ba090 Oct 30 22:07:22.817848 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.817825 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:22.820117 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.820101 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dbtlp" Oct 30 22:07:22.828038 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:22.828014 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod074eeae7_932c_4642_8fea_671898135ab1.slice/crio-89137e0ed84568ba01d3ac9b0e14fd3577d3f7a7d8a68c4b3d1a82cc43fc6601 WatchSource:0}: Error finding container 89137e0ed84568ba01d3ac9b0e14fd3577d3f7a7d8a68c4b3d1a82cc43fc6601: Status 404 returned error can't find the container with id 89137e0ed84568ba01d3ac9b0e14fd3577d3f7a7d8a68c4b3d1a82cc43fc6601 Oct 30 22:07:22.832099 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:22.832078 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11464051_0aa4_4324_b2c7_7ea2096bbed6.slice/crio-0ec135aec9bec53203b4fa8b33f029a49e2f2e4c576d202064835d2827d00338 WatchSource:0}: Error finding container 0ec135aec9bec53203b4fa8b33f029a49e2f2e4c576d202064835d2827d00338: Status 404 returned error can't find the container with id 0ec135aec9bec53203b4fa8b33f029a49e2f2e4c576d202064835d2827d00338 Oct 30 22:07:22.832636 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.832620 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zlbzf" Oct 30 22:07:22.841570 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:22.841551 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6cc017b_3f50_423a_93da_e4515336a2bc.slice/crio-81b300842b95d3636a9caf7f46ce2fbcf3b083baa3f13105be77089783c54aeb WatchSource:0}: Error finding container 81b300842b95d3636a9caf7f46ce2fbcf3b083baa3f13105be77089783c54aeb: Status 404 returned error can't find the container with id 81b300842b95d3636a9caf7f46ce2fbcf3b083baa3f13105be77089783c54aeb Oct 30 22:07:22.859808 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.859777 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-v75vr" Oct 30 22:07:22.868788 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.868769 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" Oct 30 22:07:22.868969 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:22.868951 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9381d26_e2af_4730_8b4e_06945c59585c.slice/crio-0a6c5e5091712c18a370c9096a9488012885e7100be16362bfd46acebb21aee7 WatchSource:0}: Error finding container 0a6c5e5091712c18a370c9096a9488012885e7100be16362bfd46acebb21aee7: Status 404 returned error can't find the container with id 0a6c5e5091712c18a370c9096a9488012885e7100be16362bfd46acebb21aee7 Oct 30 22:07:22.878152 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:22.878133 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5779bc8_21ad_465e_8749_8f7e9afdc03f.slice/crio-4fbf0fd61fc6b065e8ca1ffc84f42505b9d8905160540d9402b8fc358977448c WatchSource:0}: Error finding container 4fbf0fd61fc6b065e8ca1ffc84f42505b9d8905160540d9402b8fc358977448c: Status 404 returned error can't find the container with id 4fbf0fd61fc6b065e8ca1ffc84f42505b9d8905160540d9402b8fc358977448c Oct 30 22:07:22.882445 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.882428 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-f6888" Oct 30 22:07:22.891375 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:22.891355 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1f7e385_c1ea_4fd9_aff3_513ae337b5d8.slice/crio-eb20dbc00c4333ecb219b11f019b76d9cd02e2de6867ca00a83081ff0adc9951 WatchSource:0}: Error finding container eb20dbc00c4333ecb219b11f019b76d9cd02e2de6867ca00a83081ff0adc9951: Status 404 returned error can't find the container with id eb20dbc00c4333ecb219b11f019b76d9cd02e2de6867ca00a83081ff0adc9951 Oct 30 22:07:22.899629 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.899593 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-b68bn" Oct 30 22:07:22.903898 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:22.903877 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pnr6k" Oct 30 22:07:22.910045 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:22.910023 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bb239d5_4f0e_4dd6_b58b_5f3b9e014895.slice/crio-a7b56859943b84b7b61cb86d28e1f5cb2bced2688531f8d082c00371d84af8f9 WatchSource:0}: Error finding container a7b56859943b84b7b61cb86d28e1f5cb2bced2688531f8d082c00371d84af8f9: Status 404 returned error can't find the container with id a7b56859943b84b7b61cb86d28e1f5cb2bced2688531f8d082c00371d84af8f9 Oct 30 22:07:22.912744 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:07:22.912724 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b042dce_1fff_41a2_83dd_ae18f3098207.slice/crio-976467f0e8d8d1bfa25ef9bd6e507f4786388b419da2bcea0765ec9a620aba24 WatchSource:0}: Error finding container 976467f0e8d8d1bfa25ef9bd6e507f4786388b419da2bcea0765ec9a620aba24: Status 404 returned error can't find the container with id 976467f0e8d8d1bfa25ef9bd6e507f4786388b419da2bcea0765ec9a620aba24 Oct 30 22:07:22.914559 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:22.914537 2581 kuberuntime_manager.go:1707] "Unhandled Error" err="container dns-node-resolver start failed in pod node-resolver-pnr6k_openshift-dns(6b042dce-1fff-41a2-83dd-ae18f3098207): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Oct 30 22:07:22.914650 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:22.914571 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"dns-node-resolver\" with ErrImagePull: \"pull QPS exceeded\"" pod="openshift-dns/node-resolver-pnr6k" podUID="6b042dce-1fff-41a2-83dd-ae18f3098207" Oct 30 22:07:23.146723 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:23.146682 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cd46f3e8-c752-4eed-a65d-07e060bf32b4-original-pull-secret\") pod \"global-pull-secret-syncer-h9v2l\" (UID: \"cd46f3e8-c752-4eed-a65d-07e060bf32b4\") " pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:07:23.146901 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:23.146737 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65c2ef45-7211-4828-86bc-c9d7af39ec66-metrics-certs\") pod \"network-metrics-daemon-l7kkv\" (UID: \"65c2ef45-7211-4828-86bc-c9d7af39ec66\") " pod="openshift-multus/network-metrics-daemon-l7kkv" Oct 30 22:07:23.146901 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:23.146886 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 22:07:23.147023 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:23.146948 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65c2ef45-7211-4828-86bc-c9d7af39ec66-metrics-certs podName:65c2ef45-7211-4828-86bc-c9d7af39ec66 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:24.146928435 +0000 UTC m=+3.073030757 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/65c2ef45-7211-4828-86bc-c9d7af39ec66-metrics-certs") pod "network-metrics-daemon-l7kkv" (UID: "65c2ef45-7211-4828-86bc-c9d7af39ec66") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 22:07:23.147211 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:23.147186 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 22:07:23.147352 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:23.147255 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd46f3e8-c752-4eed-a65d-07e060bf32b4-original-pull-secret podName:cd46f3e8-c752-4eed-a65d-07e060bf32b4 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:24.147236505 +0000 UTC m=+3.073338815 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cd46f3e8-c752-4eed-a65d-07e060bf32b4-original-pull-secret") pod "global-pull-secret-syncer-h9v2l" (UID: "cd46f3e8-c752-4eed-a65d-07e060bf32b4") : object "kube-system"/"original-pull-secret" not registered Oct 30 22:07:23.248051 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:23.248011 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gvkg\" (UniqueName: \"kubernetes.io/projected/3155dfa9-0dc2-4256-9e16-490b3721702d-kube-api-access-2gvkg\") pod \"network-check-target-knzz5\" (UID: \"3155dfa9-0dc2-4256-9e16-490b3721702d\") " pod="openshift-network-diagnostics/network-check-target-knzz5" Oct 30 22:07:23.248278 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:23.248256 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 22:07:23.248386 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:23.248302 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 22:07:23.248386 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:23.248331 2581 projected.go:196] Error preparing data for projected volume kube-api-access-2gvkg for pod openshift-network-diagnostics/network-check-target-knzz5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 22:07:23.248492 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:23.248438 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3155dfa9-0dc2-4256-9e16-490b3721702d-kube-api-access-2gvkg podName:3155dfa9-0dc2-4256-9e16-490b3721702d nodeName:}" failed. No retries permitted until 2026-10-30 22:07:24.248371301 +0000 UTC m=+3.174473619 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-2gvkg" (UniqueName: "kubernetes.io/projected/3155dfa9-0dc2-4256-9e16-490b3721702d-kube-api-access-2gvkg") pod "network-check-target-knzz5" (UID: "3155dfa9-0dc2-4256-9e16-490b3721702d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 22:07:23.454883 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:23.454795 2581 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 22:07:23.500220 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:23.500049 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 22:02:22 +0000 UTC" deadline="2028-08-15 19:09:15.260284752 +0000 UTC" Oct 30 22:07:23.500220 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:23.500086 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15717h1m51.760203725s" Oct 30 22:07:23.594606 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:23.594542 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v75vr" event={"ID":"c9381d26-e2af-4730-8b4e-06945c59585c","Type":"ContainerStarted","Data":"0a6c5e5091712c18a370c9096a9488012885e7100be16362bfd46acebb21aee7"} Oct 30 22:07:23.613426 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:23.613390 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dbtlp" event={"ID":"11464051-0aa4-4324-b2c7-7ea2096bbed6","Type":"ContainerStarted","Data":"0ec135aec9bec53203b4fa8b33f029a49e2f2e4c576d202064835d2827d00338"} Oct 30 22:07:23.619578 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:23.619548 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal" event={"ID":"203b93f5f211a36c6aba561c49a0c709","Type":"ContainerStarted","Data":"61a90835359d781127b7c1b0c8148cf6aef8e3c7ad74148743763ac42eb9b7ad"} Oct 30 22:07:23.623032 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:23.623000 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-f6888" event={"ID":"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8","Type":"ContainerStarted","Data":"eb20dbc00c4333ecb219b11f019b76d9cd02e2de6867ca00a83081ff0adc9951"} Oct 30 22:07:23.630825 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:23.630799 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zlbzf" event={"ID":"f6cc017b-3f50-423a-93da-e4515336a2bc","Type":"ContainerStarted","Data":"81b300842b95d3636a9caf7f46ce2fbcf3b083baa3f13105be77089783c54aeb"} Oct 30 22:07:23.638785 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:23.638756 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" event={"ID":"074eeae7-932c-4642-8fea-671898135ab1","Type":"ContainerStarted","Data":"89137e0ed84568ba01d3ac9b0e14fd3577d3f7a7d8a68c4b3d1a82cc43fc6601"} Oct 30 22:07:23.653416 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:23.653384 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" event={"ID":"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44","Type":"ContainerStarted","Data":"98868e04e6322ac36774e72846ecdcde8d21ab195d25357a997c6dcead2ba090"} Oct 30 22:07:23.660441 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:23.660418 2581 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 22:07:23.674940 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:23.674912 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-24.ec2.internal" event={"ID":"7655e583c88021e605aee040501f02ec","Type":"ContainerStarted","Data":"36051ba353f4d1493e47afd3bee1fa39336698eac55c1ddd7a97c40dbd52bdfa"} Oct 30 22:07:23.681653 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:23.681629 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pnr6k" event={"ID":"6b042dce-1fff-41a2-83dd-ae18f3098207","Type":"ContainerStarted","Data":"976467f0e8d8d1bfa25ef9bd6e507f4786388b419da2bcea0765ec9a620aba24"} Oct 30 22:07:23.689668 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:23.689639 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-b68bn" event={"ID":"5bb239d5-4f0e-4dd6-b58b-5f3b9e014895","Type":"ContainerStarted","Data":"a7b56859943b84b7b61cb86d28e1f5cb2bced2688531f8d082c00371d84af8f9"} Oct 30 22:07:23.690635 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:23.690604 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"dns-node-resolver\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2a8b58f028a9a4a0345ad1f809e47fb9913d7f77c731bfd785819e1c71be9472\\\": ErrImagePull: pull QPS exceeded\"" pod="openshift-dns/node-resolver-pnr6k" podUID="6b042dce-1fff-41a2-83dd-ae18f3098207" Oct 30 22:07:23.709629 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:23.709536 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" event={"ID":"e5779bc8-21ad-465e-8749-8f7e9afdc03f","Type":"ContainerStarted","Data":"4fbf0fd61fc6b065e8ca1ffc84f42505b9d8905160540d9402b8fc358977448c"} Oct 30 22:07:24.156761 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:24.156724 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cd46f3e8-c752-4eed-a65d-07e060bf32b4-original-pull-secret\") pod \"global-pull-secret-syncer-h9v2l\" (UID: \"cd46f3e8-c752-4eed-a65d-07e060bf32b4\") " pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:07:24.156918 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:24.156786 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65c2ef45-7211-4828-86bc-c9d7af39ec66-metrics-certs\") pod \"network-metrics-daemon-l7kkv\" (UID: \"65c2ef45-7211-4828-86bc-c9d7af39ec66\") " pod="openshift-multus/network-metrics-daemon-l7kkv" Oct 30 22:07:24.156964 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:24.156946 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 22:07:24.157040 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:24.157008 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65c2ef45-7211-4828-86bc-c9d7af39ec66-metrics-certs podName:65c2ef45-7211-4828-86bc-c9d7af39ec66 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:26.156987502 +0000 UTC m=+5.083089810 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/65c2ef45-7211-4828-86bc-c9d7af39ec66-metrics-certs") pod "network-metrics-daemon-l7kkv" (UID: "65c2ef45-7211-4828-86bc-c9d7af39ec66") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 22:07:24.157447 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:24.157428 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 22:07:24.157535 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:24.157484 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd46f3e8-c752-4eed-a65d-07e060bf32b4-original-pull-secret podName:cd46f3e8-c752-4eed-a65d-07e060bf32b4 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:26.157468348 +0000 UTC m=+5.083570659 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cd46f3e8-c752-4eed-a65d-07e060bf32b4-original-pull-secret") pod "global-pull-secret-syncer-h9v2l" (UID: "cd46f3e8-c752-4eed-a65d-07e060bf32b4") : object "kube-system"/"original-pull-secret" not registered Oct 30 22:07:24.258102 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:24.258064 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gvkg\" (UniqueName: \"kubernetes.io/projected/3155dfa9-0dc2-4256-9e16-490b3721702d-kube-api-access-2gvkg\") pod \"network-check-target-knzz5\" (UID: \"3155dfa9-0dc2-4256-9e16-490b3721702d\") " pod="openshift-network-diagnostics/network-check-target-knzz5" Oct 30 22:07:24.258290 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:24.258237 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 22:07:24.258290 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:24.258279 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 22:07:24.258290 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:24.258293 2581 projected.go:196] Error preparing data for projected volume kube-api-access-2gvkg for pod openshift-network-diagnostics/network-check-target-knzz5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 22:07:24.258494 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:24.258351 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3155dfa9-0dc2-4256-9e16-490b3721702d-kube-api-access-2gvkg podName:3155dfa9-0dc2-4256-9e16-490b3721702d nodeName:}" failed. No retries permitted until 2026-10-30 22:07:26.258331013 +0000 UTC m=+5.184433323 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-2gvkg" (UniqueName: "kubernetes.io/projected/3155dfa9-0dc2-4256-9e16-490b3721702d-kube-api-access-2gvkg") pod "network-check-target-knzz5" (UID: "3155dfa9-0dc2-4256-9e16-490b3721702d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 22:07:24.500580 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:24.500463 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 22:02:22 +0000 UTC" deadline="2028-06-24 03:30:00.765902948 +0000 UTC" Oct 30 22:07:24.500580 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:24.500532 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14453h22m36.265375779s" Oct 30 22:07:24.563441 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:24.562928 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:07:24.563441 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:24.563072 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9v2l" podUID="cd46f3e8-c752-4eed-a65d-07e060bf32b4" Oct 30 22:07:24.563441 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:24.563396 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knzz5" Oct 30 22:07:24.563717 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:24.563498 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knzz5" podUID="3155dfa9-0dc2-4256-9e16-490b3721702d" Oct 30 22:07:24.563717 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:24.563534 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7kkv" Oct 30 22:07:24.563717 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:24.563621 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l7kkv" podUID="65c2ef45-7211-4828-86bc-c9d7af39ec66" Oct 30 22:07:26.173325 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:26.173246 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65c2ef45-7211-4828-86bc-c9d7af39ec66-metrics-certs\") pod \"network-metrics-daemon-l7kkv\" (UID: \"65c2ef45-7211-4828-86bc-c9d7af39ec66\") " pod="openshift-multus/network-metrics-daemon-l7kkv" Oct 30 22:07:26.173933 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:26.173357 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cd46f3e8-c752-4eed-a65d-07e060bf32b4-original-pull-secret\") pod \"global-pull-secret-syncer-h9v2l\" (UID: \"cd46f3e8-c752-4eed-a65d-07e060bf32b4\") " pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:07:26.173933 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:26.173500 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 22:07:26.173933 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:26.173569 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd46f3e8-c752-4eed-a65d-07e060bf32b4-original-pull-secret podName:cd46f3e8-c752-4eed-a65d-07e060bf32b4 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:30.17354969 +0000 UTC m=+9.099652009 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cd46f3e8-c752-4eed-a65d-07e060bf32b4-original-pull-secret") pod "global-pull-secret-syncer-h9v2l" (UID: "cd46f3e8-c752-4eed-a65d-07e060bf32b4") : object "kube-system"/"original-pull-secret" not registered Oct 30 22:07:26.173933 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:26.173638 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 22:07:26.173933 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:26.173670 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65c2ef45-7211-4828-86bc-c9d7af39ec66-metrics-certs podName:65c2ef45-7211-4828-86bc-c9d7af39ec66 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:30.173659198 +0000 UTC m=+9.099761517 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/65c2ef45-7211-4828-86bc-c9d7af39ec66-metrics-certs") pod "network-metrics-daemon-l7kkv" (UID: "65c2ef45-7211-4828-86bc-c9d7af39ec66") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 22:07:26.274271 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:26.274207 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gvkg\" (UniqueName: \"kubernetes.io/projected/3155dfa9-0dc2-4256-9e16-490b3721702d-kube-api-access-2gvkg\") pod \"network-check-target-knzz5\" (UID: \"3155dfa9-0dc2-4256-9e16-490b3721702d\") " pod="openshift-network-diagnostics/network-check-target-knzz5" Oct 30 22:07:26.274455 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:26.274388 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 22:07:26.274455 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:26.274409 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 22:07:26.274455 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:26.274422 2581 projected.go:196] Error preparing data for projected volume kube-api-access-2gvkg for pod openshift-network-diagnostics/network-check-target-knzz5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 22:07:26.274766 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:26.274481 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3155dfa9-0dc2-4256-9e16-490b3721702d-kube-api-access-2gvkg podName:3155dfa9-0dc2-4256-9e16-490b3721702d nodeName:}" failed. No retries permitted until 2026-10-30 22:07:30.274463022 +0000 UTC m=+9.200565343 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-2gvkg" (UniqueName: "kubernetes.io/projected/3155dfa9-0dc2-4256-9e16-490b3721702d-kube-api-access-2gvkg") pod "network-check-target-knzz5" (UID: "3155dfa9-0dc2-4256-9e16-490b3721702d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 22:07:26.563147 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:26.562967 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:07:26.563315 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:26.563294 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9v2l" podUID="cd46f3e8-c752-4eed-a65d-07e060bf32b4" Oct 30 22:07:26.563400 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:26.563367 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7kkv" Oct 30 22:07:26.564420 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:26.563470 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l7kkv" podUID="65c2ef45-7211-4828-86bc-c9d7af39ec66" Oct 30 22:07:26.564420 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:26.563521 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knzz5" Oct 30 22:07:26.564420 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:26.563578 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knzz5" podUID="3155dfa9-0dc2-4256-9e16-490b3721702d" Oct 30 22:07:26.730169 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:26.730054 2581 generic.go:356] "Generic (PLEG): container finished" podID="c9381d26-e2af-4730-8b4e-06945c59585c" containerID="3528fd4933a28b3cad22eae8fc5e5d8ce04a20a2ac0830f643908749a15990b8" exitCode=0 Oct 30 22:07:26.730169 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:26.730149 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v75vr" event={"ID":"c9381d26-e2af-4730-8b4e-06945c59585c","Type":"ContainerDied","Data":"3528fd4933a28b3cad22eae8fc5e5d8ce04a20a2ac0830f643908749a15990b8"} Oct 30 22:07:26.736096 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:26.736065 2581 generic.go:356] "Generic (PLEG): container finished" podID="203b93f5f211a36c6aba561c49a0c709" containerID="d8b95a78a3a21da3a8b173742807aea40466b40099b2dff7e76c6ef962e1d437" exitCode=0 Oct 30 22:07:26.736195 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:26.736115 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal" event={"ID":"203b93f5f211a36c6aba561c49a0c709","Type":"ContainerDied","Data":"d8b95a78a3a21da3a8b173742807aea40466b40099b2dff7e76c6ef962e1d437"} Oct 30 22:07:27.748824 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:27.748789 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal" event={"ID":"203b93f5f211a36c6aba561c49a0c709","Type":"ContainerStarted","Data":"c7f1875734cc72055502d36d4ad093f78d6b2d6782061c779b58ca176e028ce0"} Oct 30 22:07:27.752079 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:27.752051 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zlbzf" event={"ID":"f6cc017b-3f50-423a-93da-e4515336a2bc","Type":"ContainerStarted","Data":"2ce3f3044b4ffbc59df8acb7a252b1128f03aea99153bcf1516f47aa258609b0"} Oct 30 22:07:27.754779 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:27.754756 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-24.ec2.internal" event={"ID":"7655e583c88021e605aee040501f02ec","Type":"ContainerStarted","Data":"f583ef41b14444a12619d158a85c6ae15003034ee7466bd2ddbc5883873a5b45"} Oct 30 22:07:27.832456 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:27.832400 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal" podStartSLOduration=5.832380327 podStartE2EDuration="5.832380327s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:07:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:07:27.789370079 +0000 UTC m=+6.715472406" watchObservedRunningTime="2026-10-30 22:07:27.832380327 +0000 UTC m=+6.758482659" Oct 30 22:07:27.871310 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:27.871252 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-zlbzf" podStartSLOduration=2.919556682 podStartE2EDuration="6.871234627s" totalImagesPullingTime="3.951677945s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:07:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:07:27.833046769 +0000 UTC m=+6.759149095" watchObservedRunningTime="2026-10-30 22:07:27.871234627 +0000 UTC m=+6.797336947" Oct 30 22:07:27.871475 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:27.871358 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-24.ec2.internal" podStartSLOduration=5.87135259 podStartE2EDuration="5.87135259s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:07:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:07:27.87085372 +0000 UTC m=+6.796956044" watchObservedRunningTime="2026-10-30 22:07:27.87135259 +0000 UTC m=+6.797454915" Oct 30 22:07:28.563625 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:28.562806 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:07:28.563625 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:28.562930 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9v2l" podUID="cd46f3e8-c752-4eed-a65d-07e060bf32b4" Oct 30 22:07:28.563625 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:28.563332 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7kkv" Oct 30 22:07:28.563625 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:28.563443 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l7kkv" podUID="65c2ef45-7211-4828-86bc-c9d7af39ec66" Oct 30 22:07:28.563625 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:28.563497 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knzz5" Oct 30 22:07:28.563625 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:28.563560 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knzz5" podUID="3155dfa9-0dc2-4256-9e16-490b3721702d" Oct 30 22:07:28.760192 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:28.760156 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dbtlp" event={"ID":"11464051-0aa4-4324-b2c7-7ea2096bbed6","Type":"ContainerStarted","Data":"42c6cc773dc3a7bda3eb4852a77d7d4325d86b4314c059819abd2b7c6e34120d"} Oct 30 22:07:28.829611 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:28.829550 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-dbtlp" podStartSLOduration=2.6121772930000002 podStartE2EDuration="7.829536595s" totalImagesPullingTime="5.217359302s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:07:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:07:28.826786092 +0000 UTC m=+7.752888417" watchObservedRunningTime="2026-10-30 22:07:28.829536595 +0000 UTC m=+7.755638920" Oct 30 22:07:29.764556 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:29.764492 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" event={"ID":"e5779bc8-21ad-465e-8749-8f7e9afdc03f","Type":"ContainerStarted","Data":"be1d6ce6eade9bee3c53d3a8b941f9aaae7e5a8403ac470c33cf7c4f8c0be86c"} Oct 30 22:07:30.137106 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:30.136893 2581 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Oct 30 22:07:30.206327 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:30.206287 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cd46f3e8-c752-4eed-a65d-07e060bf32b4-original-pull-secret\") pod \"global-pull-secret-syncer-h9v2l\" (UID: \"cd46f3e8-c752-4eed-a65d-07e060bf32b4\") " pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:07:30.206473 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:30.206344 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65c2ef45-7211-4828-86bc-c9d7af39ec66-metrics-certs\") pod \"network-metrics-daemon-l7kkv\" (UID: \"65c2ef45-7211-4828-86bc-c9d7af39ec66\") " pod="openshift-multus/network-metrics-daemon-l7kkv" Oct 30 22:07:30.206545 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:30.206506 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 22:07:30.206604 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:30.206561 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65c2ef45-7211-4828-86bc-c9d7af39ec66-metrics-certs podName:65c2ef45-7211-4828-86bc-c9d7af39ec66 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:38.206542885 +0000 UTC m=+17.132645200 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/65c2ef45-7211-4828-86bc-c9d7af39ec66-metrics-certs") pod "network-metrics-daemon-l7kkv" (UID: "65c2ef45-7211-4828-86bc-c9d7af39ec66") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 22:07:30.207001 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:30.206917 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 22:07:30.207001 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:30.206965 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd46f3e8-c752-4eed-a65d-07e060bf32b4-original-pull-secret podName:cd46f3e8-c752-4eed-a65d-07e060bf32b4 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:38.206951995 +0000 UTC m=+17.133054310 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cd46f3e8-c752-4eed-a65d-07e060bf32b4-original-pull-secret") pod "global-pull-secret-syncer-h9v2l" (UID: "cd46f3e8-c752-4eed-a65d-07e060bf32b4") : object "kube-system"/"original-pull-secret" not registered Oct 30 22:07:30.307425 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:30.307343 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gvkg\" (UniqueName: \"kubernetes.io/projected/3155dfa9-0dc2-4256-9e16-490b3721702d-kube-api-access-2gvkg\") pod \"network-check-target-knzz5\" (UID: \"3155dfa9-0dc2-4256-9e16-490b3721702d\") " pod="openshift-network-diagnostics/network-check-target-knzz5" Oct 30 22:07:30.307581 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:30.307508 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 22:07:30.307581 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:30.307526 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 22:07:30.307581 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:30.307538 2581 projected.go:196] Error preparing data for projected volume kube-api-access-2gvkg for pod openshift-network-diagnostics/network-check-target-knzz5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 22:07:30.307726 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:30.307588 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3155dfa9-0dc2-4256-9e16-490b3721702d-kube-api-access-2gvkg podName:3155dfa9-0dc2-4256-9e16-490b3721702d nodeName:}" failed. No retries permitted until 2026-10-30 22:07:38.30757065 +0000 UTC m=+17.233672961 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-2gvkg" (UniqueName: "kubernetes.io/projected/3155dfa9-0dc2-4256-9e16-490b3721702d-kube-api-access-2gvkg") pod "network-check-target-knzz5" (UID: "3155dfa9-0dc2-4256-9e16-490b3721702d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 22:07:30.483198 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:30.483166 2581 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-zlbzf" Oct 30 22:07:30.483715 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:30.483691 2581 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-zlbzf" Oct 30 22:07:30.502537 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:30.502429 2581 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T22:07:30.136915486Z","UUID":"7525449d-33c1-4b0d-ad3b-5d1ba50e6413","Handler":null,"Name":"","Endpoint":""} Oct 30 22:07:30.505024 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:30.504652 2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Oct 30 22:07:30.505024 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:30.504679 2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Oct 30 22:07:30.562836 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:30.562758 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:07:30.562836 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:30.562834 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knzz5" Oct 30 22:07:30.563057 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:30.562926 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7kkv" Oct 30 22:07:30.563057 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:30.562927 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9v2l" podUID="cd46f3e8-c752-4eed-a65d-07e060bf32b4" Oct 30 22:07:30.563057 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:30.562987 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knzz5" podUID="3155dfa9-0dc2-4256-9e16-490b3721702d" Oct 30 22:07:30.563252 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:30.563070 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l7kkv" podUID="65c2ef45-7211-4828-86bc-c9d7af39ec66" Oct 30 22:07:30.768291 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:30.768258 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" event={"ID":"e5779bc8-21ad-465e-8749-8f7e9afdc03f","Type":"ContainerStarted","Data":"edb51a7272db4a65bb6233c7c58fc978d42c0328d65f97a94313568f0fddb7f9"} Oct 30 22:07:30.771901 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:30.771875 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v75vr" event={"ID":"c9381d26-e2af-4730-8b4e-06945c59585c","Type":"ContainerStarted","Data":"373a51f4307dda59159c6142c60b1cb226d82e6d46f1df486ade756a7db85d5d"} Oct 30 22:07:30.775520 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:30.775496 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-f6888" event={"ID":"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8","Type":"ContainerStarted","Data":"082295f4fbd92314a9a812df918b25fe52c097df3a193beab5074e9e163f3dea"} Oct 30 22:07:31.779381 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:31.779342 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" event={"ID":"e5779bc8-21ad-465e-8749-8f7e9afdc03f","Type":"ContainerStarted","Data":"274d0d9b100f5e17b560beda8779fc8b3298ff7cf5ed32a3c7836e143fe267d0"} Oct 30 22:07:31.781671 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:31.781640 2581 generic.go:356] "Generic (PLEG): container finished" podID="c9381d26-e2af-4730-8b4e-06945c59585c" containerID="373a51f4307dda59159c6142c60b1cb226d82e6d46f1df486ade756a7db85d5d" exitCode=0 Oct 30 22:07:31.781773 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:31.781734 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v75vr" event={"ID":"c9381d26-e2af-4730-8b4e-06945c59585c","Type":"ContainerDied","Data":"373a51f4307dda59159c6142c60b1cb226d82e6d46f1df486ade756a7db85d5d"} Oct 30 22:07:31.822028 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:31.821978 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-f6888" podStartSLOduration=2.177820198 podStartE2EDuration="9.821965472s" totalImagesPullingTime="7.644145274s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:07:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:07:30.864378831 +0000 UTC m=+9.790481156" watchObservedRunningTime="2026-10-30 22:07:31.821965472 +0000 UTC m=+10.748067796" Oct 30 22:07:31.867903 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:31.867851 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" podStartSLOduration=2.038237169 podStartE2EDuration="9.867832951s" totalImagesPullingTime="7.829595782s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:07:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:07:31.82251604 +0000 UTC m=+10.748618364" watchObservedRunningTime="2026-10-30 22:07:31.867832951 +0000 UTC m=+10.793935293" Oct 30 22:07:32.562350 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:32.562140 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:07:32.562498 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:32.562239 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knzz5" Oct 30 22:07:32.562498 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:32.562438 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9v2l" podUID="cd46f3e8-c752-4eed-a65d-07e060bf32b4" Oct 30 22:07:32.562587 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:32.562250 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7kkv" Oct 30 22:07:32.562587 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:32.562509 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knzz5" podUID="3155dfa9-0dc2-4256-9e16-490b3721702d" Oct 30 22:07:32.562653 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:32.562593 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l7kkv" podUID="65c2ef45-7211-4828-86bc-c9d7af39ec66" Oct 30 22:07:32.785510 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:32.785467 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" event={"ID":"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44","Type":"ContainerStarted","Data":"5fc0f31041b9d63f52fdb267bada160cfb6b08af9403b1a958445ebac4d14583"} Oct 30 22:07:32.786959 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:32.786927 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-b68bn" event={"ID":"5bb239d5-4f0e-4dd6-b58b-5f3b9e014895","Type":"ContainerStarted","Data":"9cdfd3300268fb75a1e6aa3195470137332d0b63b8ffe9b99fce910f98b2cb6a"} Oct 30 22:07:32.788875 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:32.788851 2581 generic.go:356] "Generic (PLEG): container finished" podID="c9381d26-e2af-4730-8b4e-06945c59585c" containerID="f5892fef9173fe1fab830db1d7a0a6deb7dd59be988e2390f945b5c3da199d30" exitCode=0 Oct 30 22:07:32.788980 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:32.788935 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v75vr" event={"ID":"c9381d26-e2af-4730-8b4e-06945c59585c","Type":"ContainerDied","Data":"f5892fef9173fe1fab830db1d7a0a6deb7dd59be988e2390f945b5c3da199d30"} Oct 30 22:07:32.837552 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:32.837510 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" podStartSLOduration=2.254402659 podStartE2EDuration="11.837496843s" totalImagesPullingTime="9.583094184s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:07:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:07:32.836993551 +0000 UTC m=+11.763096054" watchObservedRunningTime="2026-10-30 22:07:32.837496843 +0000 UTC m=+11.763599167" Oct 30 22:07:33.793504 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:33.793469 2581 generic.go:356] "Generic (PLEG): container finished" podID="c9381d26-e2af-4730-8b4e-06945c59585c" containerID="7c3a93c19db6632b6bb5575a3480ad2072ed954a14068dffd75a26f4c51ed077" exitCode=0 Oct 30 22:07:33.794066 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:33.793566 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v75vr" event={"ID":"c9381d26-e2af-4730-8b4e-06945c59585c","Type":"ContainerDied","Data":"7c3a93c19db6632b6bb5575a3480ad2072ed954a14068dffd75a26f4c51ed077"} Oct 30 22:07:33.860648 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:33.860597 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-b68bn" podStartSLOduration=4.174003621 podStartE2EDuration="11.860581902s" totalImagesPullingTime="7.686578281s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:07:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:07:32.926651849 +0000 UTC m=+11.852754174" watchObservedRunningTime="2026-10-30 22:07:33.860581902 +0000 UTC m=+12.786684225" Oct 30 22:07:34.562433 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:34.562402 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7kkv" Oct 30 22:07:34.562592 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:34.562435 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:07:34.562592 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:34.562522 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l7kkv" podUID="65c2ef45-7211-4828-86bc-c9d7af39ec66" Oct 30 22:07:34.562709 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:34.562592 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9v2l" podUID="cd46f3e8-c752-4eed-a65d-07e060bf32b4" Oct 30 22:07:34.562709 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:34.562640 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knzz5" Oct 30 22:07:34.562798 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:34.562729 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knzz5" podUID="3155dfa9-0dc2-4256-9e16-490b3721702d" Oct 30 22:07:35.801785 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:35.801743 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v75vr" event={"ID":"c9381d26-e2af-4730-8b4e-06945c59585c","Type":"ContainerStarted","Data":"10a0c9b2bbd7ee49f4ebc0f3e62b685d09bd313f36fc21b1de82b366a99b2917"} Oct 30 22:07:36.562640 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:36.562469 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knzz5" Oct 30 22:07:36.562787 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:36.562545 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7kkv" Oct 30 22:07:36.562787 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:36.562711 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knzz5" podUID="3155dfa9-0dc2-4256-9e16-490b3721702d" Oct 30 22:07:36.562787 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:36.562553 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:07:36.562882 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:36.562792 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l7kkv" podUID="65c2ef45-7211-4828-86bc-c9d7af39ec66" Oct 30 22:07:36.562882 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:36.562855 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9v2l" podUID="cd46f3e8-c752-4eed-a65d-07e060bf32b4" Oct 30 22:07:36.806327 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:36.806294 2581 generic.go:356] "Generic (PLEG): container finished" podID="c9381d26-e2af-4730-8b4e-06945c59585c" containerID="10a0c9b2bbd7ee49f4ebc0f3e62b685d09bd313f36fc21b1de82b366a99b2917" exitCode=0 Oct 30 22:07:36.806783 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:36.806331 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v75vr" event={"ID":"c9381d26-e2af-4730-8b4e-06945c59585c","Type":"ContainerDied","Data":"10a0c9b2bbd7ee49f4ebc0f3e62b685d09bd313f36fc21b1de82b366a99b2917"} Oct 30 22:07:37.811957 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:37.811723 2581 generic.go:356] "Generic (PLEG): container finished" podID="c9381d26-e2af-4730-8b4e-06945c59585c" containerID="e95cee11b495d8aa21d824e37929e77e6d965e46f61d7d2bbb52b7b5d92e031b" exitCode=0 Oct 30 22:07:37.812428 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:37.811805 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v75vr" event={"ID":"c9381d26-e2af-4730-8b4e-06945c59585c","Type":"ContainerDied","Data":"e95cee11b495d8aa21d824e37929e77e6d965e46f61d7d2bbb52b7b5d92e031b"} Oct 30 22:07:37.814618 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:37.814600 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ql6lh_074eeae7-932c-4642-8fea-671898135ab1/ovn-acl-logging/0.log" Oct 30 22:07:37.814909 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:37.814892 2581 generic.go:356] "Generic (PLEG): container finished" podID="074eeae7-932c-4642-8fea-671898135ab1" containerID="faf0c33ee470fd578edf1aeac1625ea138771e550ea0c26c6ba30af959ef8352" exitCode=1 Oct 30 22:07:37.814957 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:37.814929 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" event={"ID":"074eeae7-932c-4642-8fea-671898135ab1","Type":"ContainerStarted","Data":"01e905964bab1f5bf0d80074c5daede64a0446c972c6509f3f35b07ef133f232"} Oct 30 22:07:37.815006 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:37.814980 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" event={"ID":"074eeae7-932c-4642-8fea-671898135ab1","Type":"ContainerStarted","Data":"19e0897a7b2754733133adb94d41893727549df6b297ebe0b8410519e181cbff"} Oct 30 22:07:37.815042 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:37.815010 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" event={"ID":"074eeae7-932c-4642-8fea-671898135ab1","Type":"ContainerStarted","Data":"840f8b76d985b38ed8234c1b715cfe195098d10d3804f7a93c98814c44b17cb0"} Oct 30 22:07:37.815042 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:37.815037 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" event={"ID":"074eeae7-932c-4642-8fea-671898135ab1","Type":"ContainerStarted","Data":"25770da755f19df450e923f15aafa76766d1d5cdad2a2928f65d0b8060fa0263"} Oct 30 22:07:37.815102 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:37.815062 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" event={"ID":"074eeae7-932c-4642-8fea-671898135ab1","Type":"ContainerDied","Data":"faf0c33ee470fd578edf1aeac1625ea138771e550ea0c26c6ba30af959ef8352"} Oct 30 22:07:37.815102 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:37.815092 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" event={"ID":"074eeae7-932c-4642-8fea-671898135ab1","Type":"ContainerStarted","Data":"cca8bcc2e06af90e99f4e4c1c8e72024e089ffa8487627f810ce2ef7d764b695"} Oct 30 22:07:38.267210 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:38.267177 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cd46f3e8-c752-4eed-a65d-07e060bf32b4-original-pull-secret\") pod \"global-pull-secret-syncer-h9v2l\" (UID: \"cd46f3e8-c752-4eed-a65d-07e060bf32b4\") " pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:07:38.267210 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:38.267213 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65c2ef45-7211-4828-86bc-c9d7af39ec66-metrics-certs\") pod \"network-metrics-daemon-l7kkv\" (UID: \"65c2ef45-7211-4828-86bc-c9d7af39ec66\") " pod="openshift-multus/network-metrics-daemon-l7kkv" Oct 30 22:07:38.267404 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:38.267327 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 22:07:38.267404 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:38.267384 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd46f3e8-c752-4eed-a65d-07e060bf32b4-original-pull-secret podName:cd46f3e8-c752-4eed-a65d-07e060bf32b4 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:54.267369253 +0000 UTC m=+33.193471572 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cd46f3e8-c752-4eed-a65d-07e060bf32b4-original-pull-secret") pod "global-pull-secret-syncer-h9v2l" (UID: "cd46f3e8-c752-4eed-a65d-07e060bf32b4") : object "kube-system"/"original-pull-secret" not registered Oct 30 22:07:38.267404 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:38.267332 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 22:07:38.267497 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:38.267416 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65c2ef45-7211-4828-86bc-c9d7af39ec66-metrics-certs podName:65c2ef45-7211-4828-86bc-c9d7af39ec66 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:54.26740932 +0000 UTC m=+33.193511621 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/65c2ef45-7211-4828-86bc-c9d7af39ec66-metrics-certs") pod "network-metrics-daemon-l7kkv" (UID: "65c2ef45-7211-4828-86bc-c9d7af39ec66") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 22:07:38.368445 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:38.368415 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gvkg\" (UniqueName: \"kubernetes.io/projected/3155dfa9-0dc2-4256-9e16-490b3721702d-kube-api-access-2gvkg\") pod \"network-check-target-knzz5\" (UID: \"3155dfa9-0dc2-4256-9e16-490b3721702d\") " pod="openshift-network-diagnostics/network-check-target-knzz5" Oct 30 22:07:38.368579 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:38.368556 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 22:07:38.368579 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:38.368572 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 22:07:38.368663 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:38.368581 2581 projected.go:196] Error preparing data for projected volume kube-api-access-2gvkg for pod openshift-network-diagnostics/network-check-target-knzz5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 22:07:38.368663 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:38.368621 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3155dfa9-0dc2-4256-9e16-490b3721702d-kube-api-access-2gvkg podName:3155dfa9-0dc2-4256-9e16-490b3721702d nodeName:}" failed. No retries permitted until 2026-10-30 22:07:54.368610345 +0000 UTC m=+33.294712651 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-2gvkg" (UniqueName: "kubernetes.io/projected/3155dfa9-0dc2-4256-9e16-490b3721702d-kube-api-access-2gvkg") pod "network-check-target-knzz5" (UID: "3155dfa9-0dc2-4256-9e16-490b3721702d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 22:07:38.562930 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:38.562852 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:07:38.562930 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:38.562873 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7kkv" Oct 30 22:07:38.562930 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:38.562893 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knzz5" Oct 30 22:07:38.563127 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:38.562966 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9v2l" podUID="cd46f3e8-c752-4eed-a65d-07e060bf32b4" Oct 30 22:07:38.563127 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:38.563016 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l7kkv" podUID="65c2ef45-7211-4828-86bc-c9d7af39ec66" Oct 30 22:07:38.563127 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:38.563086 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knzz5" podUID="3155dfa9-0dc2-4256-9e16-490b3721702d" Oct 30 22:07:38.817999 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:38.817921 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pnr6k" event={"ID":"6b042dce-1fff-41a2-83dd-ae18f3098207","Type":"ContainerStarted","Data":"5aa72bb70ea8aa77f3f34fba357fa01d175b6d2c346fde2d4a438443cd361ee9"} Oct 30 22:07:38.821190 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:38.821165 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v75vr" event={"ID":"c9381d26-e2af-4730-8b4e-06945c59585c","Type":"ContainerStarted","Data":"444ff191eebcff2926f80580bcf0199153d4779ac1eae57633e6cc39add47527"} Oct 30 22:07:38.858857 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:38.858819 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-pnr6k" podStartSLOduration=1.20808135 podStartE2EDuration="16.858805408s" totalImagesPullingTime="15.650724058s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:07:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:07:38.858497742 +0000 UTC m=+17.784600067" watchObservedRunningTime="2026-10-30 22:07:38.858805408 +0000 UTC m=+17.784907731" Oct 30 22:07:38.915974 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:38.915935 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-v75vr" podStartSLOduration=7.327950214 podStartE2EDuration="16.915923212s" totalImagesPullingTime="9.587972998s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:07:22 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:07:38.903865287 +0000 UTC m=+17.829967610" watchObservedRunningTime="2026-10-30 22:07:38.915923212 +0000 UTC m=+17.842025535" Oct 30 22:07:39.826029 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:39.826000 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ql6lh_074eeae7-932c-4642-8fea-671898135ab1/ovn-acl-logging/0.log" Oct 30 22:07:39.826430 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:39.826409 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" event={"ID":"074eeae7-932c-4642-8fea-671898135ab1","Type":"ContainerStarted","Data":"6e47e9da9503e0920e37ee4506ca2cd2ac01bd5798563586e96e03cfb9b8379d"} Oct 30 22:07:40.562435 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:40.562403 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7kkv" Oct 30 22:07:40.562435 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:40.562433 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:07:40.562624 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:40.562452 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knzz5" Oct 30 22:07:40.562624 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:40.562557 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l7kkv" podUID="65c2ef45-7211-4828-86bc-c9d7af39ec66" Oct 30 22:07:40.562679 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:40.562660 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9v2l" podUID="cd46f3e8-c752-4eed-a65d-07e060bf32b4" Oct 30 22:07:40.562729 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:40.562716 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knzz5" podUID="3155dfa9-0dc2-4256-9e16-490b3721702d" Oct 30 22:07:41.218263 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:41.218230 2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-zlbzf" Oct 30 22:07:41.218701 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:41.218343 2581 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 22:07:41.218902 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:41.218885 2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-zlbzf" Oct 30 22:07:41.834031 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:41.833818 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ql6lh_074eeae7-932c-4642-8fea-671898135ab1/ovn-acl-logging/0.log" Oct 30 22:07:41.834346 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:41.834314 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" event={"ID":"074eeae7-932c-4642-8fea-671898135ab1","Type":"ContainerStarted","Data":"bcee4fbf60970b9b254bc014aecb67542f7f5d6690426b69a16d977ca151fc59"} Oct 30 22:07:41.834702 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:41.834684 2581 scope.go:122] "RemoveContainer" containerID="faf0c33ee470fd578edf1aeac1625ea138771e550ea0c26c6ba30af959ef8352" Oct 30 22:07:42.562139 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:42.562108 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:07:42.562139 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:42.562143 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knzz5" Oct 30 22:07:42.562139 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:42.562139 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7kkv" Oct 30 22:07:42.562839 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:42.562236 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9v2l" podUID="cd46f3e8-c752-4eed-a65d-07e060bf32b4" Oct 30 22:07:42.562839 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:42.562335 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knzz5" podUID="3155dfa9-0dc2-4256-9e16-490b3721702d" Oct 30 22:07:42.562839 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:42.562406 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l7kkv" podUID="65c2ef45-7211-4828-86bc-c9d7af39ec66" Oct 30 22:07:42.840310 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:42.840239 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ql6lh_074eeae7-932c-4642-8fea-671898135ab1/ovn-acl-logging/0.log" Oct 30 22:07:42.840598 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:42.840575 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" event={"ID":"074eeae7-932c-4642-8fea-671898135ab1","Type":"ContainerStarted","Data":"116491f32e634dd335e508ef384229af72f62790874fc4f70d7d9bce58f1997a"} Oct 30 22:07:42.840711 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:42.840662 2581 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 22:07:42.840815 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:42.840803 2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:42.840876 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:42.840855 2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:42.855608 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:42.855585 2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:42.855739 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:42.855680 2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:42.930467 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:42.930424 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" podStartSLOduration=7.825498133 podStartE2EDuration="21.930411623s" totalImagesPullingTime="14.10491349s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:07:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:07:42.918532674 +0000 UTC m=+21.844634998" watchObservedRunningTime="2026-10-30 22:07:42.930411623 +0000 UTC m=+21.856513925" Oct 30 22:07:43.843082 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:43.843051 2581 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 22:07:44.080453 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:44.080420 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pnr6k_6b042dce-1fff-41a2-83dd-ae18f3098207/dns-node-resolver/0.log" Oct 30 22:07:44.562214 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:44.562183 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7kkv" Oct 30 22:07:44.562399 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:44.562310 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:07:44.562399 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:44.562324 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knzz5" Oct 30 22:07:44.562399 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:44.562327 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l7kkv" podUID="65c2ef45-7211-4828-86bc-c9d7af39ec66" Oct 30 22:07:44.562399 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:44.562392 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9v2l" podUID="cd46f3e8-c752-4eed-a65d-07e060bf32b4" Oct 30 22:07:44.562553 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:44.562444 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knzz5" podUID="3155dfa9-0dc2-4256-9e16-490b3721702d" Oct 30 22:07:44.603736 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:44.603709 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dbtlp_11464051-0aa4-4324-b2c7-7ea2096bbed6/node-ca/0.log" Oct 30 22:07:44.844889 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:44.844816 2581 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 22:07:46.562605 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:46.562580 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7kkv" Oct 30 22:07:46.563042 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:46.562580 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:07:46.563042 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:46.562678 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l7kkv" podUID="65c2ef45-7211-4828-86bc-c9d7af39ec66" Oct 30 22:07:46.563042 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:46.562580 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knzz5" Oct 30 22:07:46.563042 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:46.562757 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9v2l" podUID="cd46f3e8-c752-4eed-a65d-07e060bf32b4" Oct 30 22:07:46.563042 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:46.562806 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knzz5" podUID="3155dfa9-0dc2-4256-9e16-490b3721702d" Oct 30 22:07:47.389452 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:47.389413 2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:07:47.389645 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:47.389635 2581 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 22:07:47.399075 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:47.399042 2581 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" podUID="074eeae7-932c-4642-8fea-671898135ab1" containerName="ovnkube-controller" probeResult="failure" output="" Oct 30 22:07:47.407144 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:47.407104 2581 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" podUID="074eeae7-932c-4642-8fea-671898135ab1" containerName="ovnkube-controller" probeResult="failure" output="" Oct 30 22:07:48.562146 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:48.562112 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knzz5" Oct 30 22:07:48.562547 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:48.562159 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7kkv" Oct 30 22:07:48.562547 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:48.562112 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:07:48.562547 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:48.562208 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knzz5" podUID="3155dfa9-0dc2-4256-9e16-490b3721702d" Oct 30 22:07:48.562547 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:48.562297 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9v2l" podUID="cd46f3e8-c752-4eed-a65d-07e060bf32b4" Oct 30 22:07:48.562547 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:48.562371 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l7kkv" podUID="65c2ef45-7211-4828-86bc-c9d7af39ec66" Oct 30 22:07:50.562134 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:50.562110 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7kkv" Oct 30 22:07:50.562546 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:50.562112 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knzz5" Oct 30 22:07:50.562546 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:50.562204 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l7kkv" podUID="65c2ef45-7211-4828-86bc-c9d7af39ec66" Oct 30 22:07:50.562546 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:50.562110 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:07:50.562546 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:50.562290 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knzz5" podUID="3155dfa9-0dc2-4256-9e16-490b3721702d" Oct 30 22:07:50.562546 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:50.562361 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9v2l" podUID="cd46f3e8-c752-4eed-a65d-07e060bf32b4" Oct 30 22:07:52.562385 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:52.562352 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7kkv" Oct 30 22:07:52.562857 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:52.562424 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:07:52.562857 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:52.562452 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knzz5" Oct 30 22:07:52.562857 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:52.562538 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9v2l" podUID="cd46f3e8-c752-4eed-a65d-07e060bf32b4" Oct 30 22:07:52.562857 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:52.562603 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knzz5" podUID="3155dfa9-0dc2-4256-9e16-490b3721702d" Oct 30 22:07:52.562857 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:52.562676 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l7kkv" podUID="65c2ef45-7211-4828-86bc-c9d7af39ec66" Oct 30 22:07:54.283517 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:54.283486 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cd46f3e8-c752-4eed-a65d-07e060bf32b4-original-pull-secret\") pod \"global-pull-secret-syncer-h9v2l\" (UID: \"cd46f3e8-c752-4eed-a65d-07e060bf32b4\") " pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:07:54.283517 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:54.283522 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65c2ef45-7211-4828-86bc-c9d7af39ec66-metrics-certs\") pod \"network-metrics-daemon-l7kkv\" (UID: \"65c2ef45-7211-4828-86bc-c9d7af39ec66\") " pod="openshift-multus/network-metrics-daemon-l7kkv" Oct 30 22:07:54.283948 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:54.283632 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 22:07:54.283948 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:54.283688 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd46f3e8-c752-4eed-a65d-07e060bf32b4-original-pull-secret podName:cd46f3e8-c752-4eed-a65d-07e060bf32b4 nodeName:}" failed. No retries permitted until 2026-10-30 22:08:26.283674089 +0000 UTC m=+65.209776396 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cd46f3e8-c752-4eed-a65d-07e060bf32b4-original-pull-secret") pod "global-pull-secret-syncer-h9v2l" (UID: "cd46f3e8-c752-4eed-a65d-07e060bf32b4") : object "kube-system"/"original-pull-secret" not registered Oct 30 22:07:54.283948 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:54.283632 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 22:07:54.283948 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:54.283756 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65c2ef45-7211-4828-86bc-c9d7af39ec66-metrics-certs podName:65c2ef45-7211-4828-86bc-c9d7af39ec66 nodeName:}" failed. No retries permitted until 2026-10-30 22:08:26.283745444 +0000 UTC m=+65.209847746 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/65c2ef45-7211-4828-86bc-c9d7af39ec66-metrics-certs") pod "network-metrics-daemon-l7kkv" (UID: "65c2ef45-7211-4828-86bc-c9d7af39ec66") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 22:07:54.384792 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:54.384755 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gvkg\" (UniqueName: \"kubernetes.io/projected/3155dfa9-0dc2-4256-9e16-490b3721702d-kube-api-access-2gvkg\") pod \"network-check-target-knzz5\" (UID: \"3155dfa9-0dc2-4256-9e16-490b3721702d\") " pod="openshift-network-diagnostics/network-check-target-knzz5" Oct 30 22:07:54.384965 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:54.384893 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 22:07:54.384965 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:54.384913 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 22:07:54.384965 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:54.384926 2581 projected.go:196] Error preparing data for projected volume kube-api-access-2gvkg for pod openshift-network-diagnostics/network-check-target-knzz5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 22:07:54.385119 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:54.384979 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3155dfa9-0dc2-4256-9e16-490b3721702d-kube-api-access-2gvkg podName:3155dfa9-0dc2-4256-9e16-490b3721702d nodeName:}" failed. No retries permitted until 2026-10-30 22:08:26.384963105 +0000 UTC m=+65.311065407 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-2gvkg" (UniqueName: "kubernetes.io/projected/3155dfa9-0dc2-4256-9e16-490b3721702d-kube-api-access-2gvkg") pod "network-check-target-knzz5" (UID: "3155dfa9-0dc2-4256-9e16-490b3721702d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 22:07:54.562842 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:54.562779 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:07:54.562959 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:54.562780 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knzz5" Oct 30 22:07:54.562959 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:54.562780 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7kkv" Oct 30 22:07:54.563032 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:54.562953 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knzz5" podUID="3155dfa9-0dc2-4256-9e16-490b3721702d" Oct 30 22:07:54.563032 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:54.562866 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9v2l" podUID="cd46f3e8-c752-4eed-a65d-07e060bf32b4" Oct 30 22:07:54.563098 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:54.563028 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l7kkv" podUID="65c2ef45-7211-4828-86bc-c9d7af39ec66" Oct 30 22:07:56.562375 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:56.562345 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7kkv" Oct 30 22:07:56.562736 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:56.562398 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:07:56.562736 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:56.562419 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knzz5" Oct 30 22:07:56.562736 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:56.562525 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9v2l" podUID="cd46f3e8-c752-4eed-a65d-07e060bf32b4" Oct 30 22:07:56.562736 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:56.562558 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knzz5" podUID="3155dfa9-0dc2-4256-9e16-490b3721702d" Oct 30 22:07:56.562736 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:56.562629 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l7kkv" podUID="65c2ef45-7211-4828-86bc-c9d7af39ec66" Oct 30 22:07:58.562489 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:58.562460 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knzz5" Oct 30 22:07:58.562960 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:58.562545 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:07:58.562960 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:07:58.562563 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7kkv" Oct 30 22:07:58.562960 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:58.562549 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knzz5" podUID="3155dfa9-0dc2-4256-9e16-490b3721702d" Oct 30 22:07:58.562960 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:58.562663 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9v2l" podUID="cd46f3e8-c752-4eed-a65d-07e060bf32b4" Oct 30 22:07:58.562960 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:07:58.562694 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l7kkv" podUID="65c2ef45-7211-4828-86bc-c9d7af39ec66" Oct 30 22:08:00.562492 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:00.562468 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:08:00.562492 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:00.562477 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knzz5" Oct 30 22:08:00.563012 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:00.562469 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7kkv" Oct 30 22:08:00.563012 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:08:00.562553 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9v2l" podUID="cd46f3e8-c752-4eed-a65d-07e060bf32b4" Oct 30 22:08:00.563012 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:08:00.562629 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l7kkv" podUID="65c2ef45-7211-4828-86bc-c9d7af39ec66" Oct 30 22:08:00.563012 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:08:00.562683 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knzz5" podUID="3155dfa9-0dc2-4256-9e16-490b3721702d" Oct 30 22:08:02.562120 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:02.562091 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:08:02.562120 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:02.562108 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7kkv" Oct 30 22:08:02.562643 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:02.562104 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knzz5" Oct 30 22:08:02.562643 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:08:02.562198 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9v2l" podUID="cd46f3e8-c752-4eed-a65d-07e060bf32b4" Oct 30 22:08:02.562643 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:08:02.562281 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knzz5" podUID="3155dfa9-0dc2-4256-9e16-490b3721702d" Oct 30 22:08:02.562643 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:08:02.562347 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l7kkv" podUID="65c2ef45-7211-4828-86bc-c9d7af39ec66" Oct 30 22:08:02.689832 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:02.689799 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h9v2l"] Oct 30 22:08:02.694326 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:02.694300 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-l7kkv"] Oct 30 22:08:02.694448 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:02.694367 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-knzz5"] Oct 30 22:08:02.880239 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:02.880065 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7kkv" Oct 30 22:08:02.880414 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:02.880064 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knzz5" Oct 30 22:08:02.880414 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:08:02.880350 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l7kkv" podUID="65c2ef45-7211-4828-86bc-c9d7af39ec66" Oct 30 22:08:02.880414 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:02.880073 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:08:02.880414 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:08:02.880387 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knzz5" podUID="3155dfa9-0dc2-4256-9e16-490b3721702d" Oct 30 22:08:02.880630 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:08:02.880449 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9v2l" podUID="cd46f3e8-c752-4eed-a65d-07e060bf32b4" Oct 30 22:08:04.562573 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:04.562538 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7kkv" Oct 30 22:08:04.562573 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:04.562559 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knzz5" Oct 30 22:08:04.563030 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:08:04.562656 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l7kkv" podUID="65c2ef45-7211-4828-86bc-c9d7af39ec66" Oct 30 22:08:04.563030 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:04.562719 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:08:04.563030 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:08:04.562812 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9v2l" podUID="cd46f3e8-c752-4eed-a65d-07e060bf32b4" Oct 30 22:08:04.563030 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:08:04.563022 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knzz5" podUID="3155dfa9-0dc2-4256-9e16-490b3721702d" Oct 30 22:08:06.562379 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:06.562352 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knzz5" Oct 30 22:08:06.562841 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:06.562385 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7kkv" Oct 30 22:08:06.562841 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:08:06.562442 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-knzz5" podUID="3155dfa9-0dc2-4256-9e16-490b3721702d" Oct 30 22:08:06.562841 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:06.562482 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:08:06.562841 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:08:06.562559 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9v2l" podUID="cd46f3e8-c752-4eed-a65d-07e060bf32b4" Oct 30 22:08:06.562841 ip-10-0-142-24 kubenswrapper[2581]: E1030 22:08:06.562609 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l7kkv" podUID="65c2ef45-7211-4828-86bc-c9d7af39ec66" Oct 30 22:08:07.327904 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.327834 2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-24.ec2.internal" event="NodeReady" Oct 30 22:08:07.328041 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.327969 2581 kubelet_node_status.go:477] "Fast updating node status as it just became ready" Oct 30 22:08:07.389878 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.389847 2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-pl4x2"] Oct 30 22:08:07.391541 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.391525 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pl4x2" Oct 30 22:08:07.393266 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.393239 2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-9mkk7"] Oct 30 22:08:07.394679 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.394658 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9mkk7" Oct 30 22:08:07.406712 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.406694 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hk5lx\"" Oct 30 22:08:07.406821 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.406742 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Oct 30 22:08:07.406821 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.406754 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Oct 30 22:08:07.414307 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.414288 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pl4x2"] Oct 30 22:08:07.417338 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.417286 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9mkk7"] Oct 30 22:08:07.481576 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.481542 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cj545\" (UniqueName: \"kubernetes.io/projected/c373c767-d743-4819-be7c-4c9c830eb9db-kube-api-access-cj545\") pod \"dns-default-pl4x2\" (UID: \"c373c767-d743-4819-be7c-4c9c830eb9db\") " pod="openshift-dns/dns-default-pl4x2" Oct 30 22:08:07.481739 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.481586 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjk5b\" (UniqueName: \"kubernetes.io/projected/f64980b7-13c0-40c3-aa3f-277916f24518-kube-api-access-tjk5b\") pod \"ingress-canary-9mkk7\" (UID: \"f64980b7-13c0-40c3-aa3f-277916f24518\") " pod="openshift-ingress-canary/ingress-canary-9mkk7" Oct 30 22:08:07.481739 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.481613 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c373c767-d743-4819-be7c-4c9c830eb9db-config-volume\") pod \"dns-default-pl4x2\" (UID: \"c373c767-d743-4819-be7c-4c9c830eb9db\") " pod="openshift-dns/dns-default-pl4x2" Oct 30 22:08:07.481739 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.481651 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c373c767-d743-4819-be7c-4c9c830eb9db-metrics-tls\") pod \"dns-default-pl4x2\" (UID: \"c373c767-d743-4819-be7c-4c9c830eb9db\") " pod="openshift-dns/dns-default-pl4x2" Oct 30 22:08:07.481857 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.481777 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c373c767-d743-4819-be7c-4c9c830eb9db-tmp-dir\") pod \"dns-default-pl4x2\" (UID: \"c373c767-d743-4819-be7c-4c9c830eb9db\") " pod="openshift-dns/dns-default-pl4x2" Oct 30 22:08:07.481857 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.481812 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f64980b7-13c0-40c3-aa3f-277916f24518-cert\") pod \"ingress-canary-9mkk7\" (UID: \"f64980b7-13c0-40c3-aa3f-277916f24518\") " pod="openshift-ingress-canary/ingress-canary-9mkk7" Oct 30 22:08:07.494310 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.494281 2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-pn4kg"] Oct 30 22:08:07.496032 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.496017 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pn4kg" Oct 30 22:08:07.507191 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.507170 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-7vf6q\"" Oct 30 22:08:07.507191 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.507174 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Oct 30 22:08:07.507343 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.507173 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Oct 30 22:08:07.507343 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.507235 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Oct 30 22:08:07.520122 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.520102 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pn4kg"] Oct 30 22:08:07.582715 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.582645 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjk5b\" (UniqueName: \"kubernetes.io/projected/f64980b7-13c0-40c3-aa3f-277916f24518-kube-api-access-tjk5b\") pod \"ingress-canary-9mkk7\" (UID: \"f64980b7-13c0-40c3-aa3f-277916f24518\") " pod="openshift-ingress-canary/ingress-canary-9mkk7" Oct 30 22:08:07.583112 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.583050 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c373c767-d743-4819-be7c-4c9c830eb9db-config-volume\") pod \"dns-default-pl4x2\" (UID: \"c373c767-d743-4819-be7c-4c9c830eb9db\") " pod="openshift-dns/dns-default-pl4x2" Oct 30 22:08:07.583168 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.583111 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c373c767-d743-4819-be7c-4c9c830eb9db-metrics-tls\") pod \"dns-default-pl4x2\" (UID: \"c373c767-d743-4819-be7c-4c9c830eb9db\") " pod="openshift-dns/dns-default-pl4x2" Oct 30 22:08:07.583266 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.583245 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5dfaf077-3637-4cc9-8681-466c4e2cc5f4-crio-socket\") pod \"insights-runtime-extractor-pn4kg\" (UID: \"5dfaf077-3637-4cc9-8681-466c4e2cc5f4\") " pod="openshift-insights/insights-runtime-extractor-pn4kg" Oct 30 22:08:07.583338 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.583327 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c373c767-d743-4819-be7c-4c9c830eb9db-tmp-dir\") pod \"dns-default-pl4x2\" (UID: \"c373c767-d743-4819-be7c-4c9c830eb9db\") " pod="openshift-dns/dns-default-pl4x2" Oct 30 22:08:07.583389 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.583366 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5dfaf077-3637-4cc9-8681-466c4e2cc5f4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pn4kg\" (UID: \"5dfaf077-3637-4cc9-8681-466c4e2cc5f4\") " pod="openshift-insights/insights-runtime-extractor-pn4kg" Oct 30 22:08:07.583447 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.583422 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f64980b7-13c0-40c3-aa3f-277916f24518-cert\") pod \"ingress-canary-9mkk7\" (UID: \"f64980b7-13c0-40c3-aa3f-277916f24518\") " pod="openshift-ingress-canary/ingress-canary-9mkk7" Oct 30 22:08:07.583500 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.583457 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5dfaf077-3637-4cc9-8681-466c4e2cc5f4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pn4kg\" (UID: \"5dfaf077-3637-4cc9-8681-466c4e2cc5f4\") " pod="openshift-insights/insights-runtime-extractor-pn4kg" Oct 30 22:08:07.583549 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.583499 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8r24v\" (UniqueName: \"kubernetes.io/projected/5dfaf077-3637-4cc9-8681-466c4e2cc5f4-kube-api-access-8r24v\") pod \"insights-runtime-extractor-pn4kg\" (UID: \"5dfaf077-3637-4cc9-8681-466c4e2cc5f4\") " pod="openshift-insights/insights-runtime-extractor-pn4kg" Oct 30 22:08:07.583549 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.583536 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cj545\" (UniqueName: \"kubernetes.io/projected/c373c767-d743-4819-be7c-4c9c830eb9db-kube-api-access-cj545\") pod \"dns-default-pl4x2\" (UID: \"c373c767-d743-4819-be7c-4c9c830eb9db\") " pod="openshift-dns/dns-default-pl4x2" Oct 30 22:08:07.583637 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.583566 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5dfaf077-3637-4cc9-8681-466c4e2cc5f4-data-volume\") pod \"insights-runtime-extractor-pn4kg\" (UID: \"5dfaf077-3637-4cc9-8681-466c4e2cc5f4\") " pod="openshift-insights/insights-runtime-extractor-pn4kg" Oct 30 22:08:07.583934 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.583791 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c373c767-d743-4819-be7c-4c9c830eb9db-tmp-dir\") pod \"dns-default-pl4x2\" (UID: \"c373c767-d743-4819-be7c-4c9c830eb9db\") " pod="openshift-dns/dns-default-pl4x2" Oct 30 22:08:07.586241 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.584647 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c373c767-d743-4819-be7c-4c9c830eb9db-config-volume\") pod \"dns-default-pl4x2\" (UID: \"c373c767-d743-4819-be7c-4c9c830eb9db\") " pod="openshift-dns/dns-default-pl4x2" Oct 30 22:08:07.588300 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.588277 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c373c767-d743-4819-be7c-4c9c830eb9db-metrics-tls\") pod \"dns-default-pl4x2\" (UID: \"c373c767-d743-4819-be7c-4c9c830eb9db\") " pod="openshift-dns/dns-default-pl4x2" Oct 30 22:08:07.588300 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.588295 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f64980b7-13c0-40c3-aa3f-277916f24518-cert\") pod \"ingress-canary-9mkk7\" (UID: \"f64980b7-13c0-40c3-aa3f-277916f24518\") " pod="openshift-ingress-canary/ingress-canary-9mkk7" Oct 30 22:08:07.606962 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.606940 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjk5b\" (UniqueName: \"kubernetes.io/projected/f64980b7-13c0-40c3-aa3f-277916f24518-kube-api-access-tjk5b\") pod \"ingress-canary-9mkk7\" (UID: \"f64980b7-13c0-40c3-aa3f-277916f24518\") " pod="openshift-ingress-canary/ingress-canary-9mkk7" Oct 30 22:08:07.607120 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.607101 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cj545\" (UniqueName: \"kubernetes.io/projected/c373c767-d743-4819-be7c-4c9c830eb9db-kube-api-access-cj545\") pod \"dns-default-pl4x2\" (UID: \"c373c767-d743-4819-be7c-4c9c830eb9db\") " pod="openshift-dns/dns-default-pl4x2" Oct 30 22:08:07.607342 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.607325 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Oct 30 22:08:07.607446 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.607433 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Oct 30 22:08:07.607446 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.607437 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Oct 30 22:08:07.607525 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.607487 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Oct 30 22:08:07.607570 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.607542 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-r7nlg\"" Oct 30 22:08:07.685203 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.685163 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5dfaf077-3637-4cc9-8681-466c4e2cc5f4-data-volume\") pod \"insights-runtime-extractor-pn4kg\" (UID: \"5dfaf077-3637-4cc9-8681-466c4e2cc5f4\") " pod="openshift-insights/insights-runtime-extractor-pn4kg" Oct 30 22:08:07.685341 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.685258 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5dfaf077-3637-4cc9-8681-466c4e2cc5f4-crio-socket\") pod \"insights-runtime-extractor-pn4kg\" (UID: \"5dfaf077-3637-4cc9-8681-466c4e2cc5f4\") " pod="openshift-insights/insights-runtime-extractor-pn4kg" Oct 30 22:08:07.685341 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.685297 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5dfaf077-3637-4cc9-8681-466c4e2cc5f4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pn4kg\" (UID: \"5dfaf077-3637-4cc9-8681-466c4e2cc5f4\") " pod="openshift-insights/insights-runtime-extractor-pn4kg" Oct 30 22:08:07.685403 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.685360 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5dfaf077-3637-4cc9-8681-466c4e2cc5f4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pn4kg\" (UID: \"5dfaf077-3637-4cc9-8681-466c4e2cc5f4\") " pod="openshift-insights/insights-runtime-extractor-pn4kg" Oct 30 22:08:07.685403 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.685395 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8r24v\" (UniqueName: \"kubernetes.io/projected/5dfaf077-3637-4cc9-8681-466c4e2cc5f4-kube-api-access-8r24v\") pod \"insights-runtime-extractor-pn4kg\" (UID: \"5dfaf077-3637-4cc9-8681-466c4e2cc5f4\") " pod="openshift-insights/insights-runtime-extractor-pn4kg" Oct 30 22:08:07.685530 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.685508 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5dfaf077-3637-4cc9-8681-466c4e2cc5f4-crio-socket\") pod \"insights-runtime-extractor-pn4kg\" (UID: \"5dfaf077-3637-4cc9-8681-466c4e2cc5f4\") " pod="openshift-insights/insights-runtime-extractor-pn4kg" Oct 30 22:08:07.685591 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.685574 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5dfaf077-3637-4cc9-8681-466c4e2cc5f4-data-volume\") pod \"insights-runtime-extractor-pn4kg\" (UID: \"5dfaf077-3637-4cc9-8681-466c4e2cc5f4\") " pod="openshift-insights/insights-runtime-extractor-pn4kg" Oct 30 22:08:07.685848 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.685823 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5dfaf077-3637-4cc9-8681-466c4e2cc5f4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pn4kg\" (UID: \"5dfaf077-3637-4cc9-8681-466c4e2cc5f4\") " pod="openshift-insights/insights-runtime-extractor-pn4kg" Oct 30 22:08:07.687537 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.687512 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5dfaf077-3637-4cc9-8681-466c4e2cc5f4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pn4kg\" (UID: \"5dfaf077-3637-4cc9-8681-466c4e2cc5f4\") " pod="openshift-insights/insights-runtime-extractor-pn4kg" Oct 30 22:08:07.703360 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.703341 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pl4x2" Oct 30 22:08:07.707547 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.707518 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9mkk7" Oct 30 22:08:07.709368 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.709352 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8r24v\" (UniqueName: \"kubernetes.io/projected/5dfaf077-3637-4cc9-8681-466c4e2cc5f4-kube-api-access-8r24v\") pod \"insights-runtime-extractor-pn4kg\" (UID: \"5dfaf077-3637-4cc9-8681-466c4e2cc5f4\") " pod="openshift-insights/insights-runtime-extractor-pn4kg" Oct 30 22:08:07.806899 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:07.806877 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pn4kg" Oct 30 22:08:08.025102 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:08.025072 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9mkk7"] Oct 30 22:08:08.026886 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:08:08.026862 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf64980b7_13c0_40c3_aa3f_277916f24518.slice/crio-50767516a7f3953ef0ef84dec6c641854ecd806e5f5a3b2c0f75d7e46408cc12 WatchSource:0}: Error finding container 50767516a7f3953ef0ef84dec6c641854ecd806e5f5a3b2c0f75d7e46408cc12: Status 404 returned error can't find the container with id 50767516a7f3953ef0ef84dec6c641854ecd806e5f5a3b2c0f75d7e46408cc12 Oct 30 22:08:08.028152 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:08.028091 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pl4x2"] Oct 30 22:08:08.035772 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:08:08.035747 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc373c767_d743_4819_be7c_4c9c830eb9db.slice/crio-e77edf5e5faa5574836fffe4f9ac84164a97e19598785877baa1d628cf1b0a58 WatchSource:0}: Error finding container e77edf5e5faa5574836fffe4f9ac84164a97e19598785877baa1d628cf1b0a58: Status 404 returned error can't find the container with id e77edf5e5faa5574836fffe4f9ac84164a97e19598785877baa1d628cf1b0a58 Oct 30 22:08:08.098668 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:08.098645 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pn4kg"] Oct 30 22:08:08.100434 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:08:08.100382 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dfaf077_3637_4cc9_8681_466c4e2cc5f4.slice/crio-62c19734cc22ee07d6a2c049f1ee538a83cb1a6158f7b1dde8efdf1c7f7767f6 WatchSource:0}: Error finding container 62c19734cc22ee07d6a2c049f1ee538a83cb1a6158f7b1dde8efdf1c7f7767f6: Status 404 returned error can't find the container with id 62c19734cc22ee07d6a2c049f1ee538a83cb1a6158f7b1dde8efdf1c7f7767f6 Oct 30 22:08:08.562712 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:08.562678 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7kkv" Oct 30 22:08:08.562891 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:08.562715 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knzz5" Oct 30 22:08:08.562891 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:08.562719 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:08:08.578653 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:08.578625 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Oct 30 22:08:08.590591 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:08.590569 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-hmbsc\"" Oct 30 22:08:08.590985 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:08.590632 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Oct 30 22:08:08.590985 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:08.590671 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5nx2r\"" Oct 30 22:08:08.590985 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:08.590709 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-gq78g\"" Oct 30 22:08:08.590985 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:08.590738 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Oct 30 22:08:08.590985 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:08.590768 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Oct 30 22:08:08.897136 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:08.897085 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9mkk7" event={"ID":"f64980b7-13c0-40c3-aa3f-277916f24518","Type":"ContainerStarted","Data":"50767516a7f3953ef0ef84dec6c641854ecd806e5f5a3b2c0f75d7e46408cc12"} Oct 30 22:08:08.899294 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:08.899263 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pn4kg" event={"ID":"5dfaf077-3637-4cc9-8681-466c4e2cc5f4","Type":"ContainerStarted","Data":"b0478eeadf6135365185ffe7c589649d921a1a20a77cc9e0a376382352643f54"} Oct 30 22:08:08.899401 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:08.899330 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pn4kg" event={"ID":"5dfaf077-3637-4cc9-8681-466c4e2cc5f4","Type":"ContainerStarted","Data":"62c19734cc22ee07d6a2c049f1ee538a83cb1a6158f7b1dde8efdf1c7f7767f6"} Oct 30 22:08:08.900806 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:08.900784 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pl4x2" event={"ID":"c373c767-d743-4819-be7c-4c9c830eb9db","Type":"ContainerStarted","Data":"e77edf5e5faa5574836fffe4f9ac84164a97e19598785877baa1d628cf1b0a58"} Oct 30 22:08:09.109416 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.109330 2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-xs8x9"] Oct 30 22:08:09.111211 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.111188 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-xs8x9" Oct 30 22:08:09.131202 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.131176 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-b6lz9\"" Oct 30 22:08:09.131202 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.131194 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Oct 30 22:08:09.131433 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.131236 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Oct 30 22:08:09.131433 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.131271 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Oct 30 22:08:09.131433 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.131391 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Oct 30 22:08:09.131591 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.131434 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Oct 30 22:08:09.143332 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.143301 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-xs8x9"] Oct 30 22:08:09.171564 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.171535 2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-6krqs"] Oct 30 22:08:09.173828 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.173800 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6krqs" Oct 30 22:08:09.196831 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.196804 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45l5v\" (UniqueName: \"kubernetes.io/projected/df6b6cae-173c-461c-9195-4dce90bb6656-kube-api-access-45l5v\") pod \"kube-state-metrics-5dc8d7d89b-6krqs\" (UID: \"df6b6cae-173c-461c-9195-4dce90bb6656\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6krqs" Oct 30 22:08:09.196992 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.196847 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zxs5\" (UniqueName: \"kubernetes.io/projected/e129700d-0583-4316-bcee-431c57b0effa-kube-api-access-5zxs5\") pod \"openshift-state-metrics-7bd7489957-xs8x9\" (UID: \"e129700d-0583-4316-bcee-431c57b0effa\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-xs8x9" Oct 30 22:08:09.196992 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.196868 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/df6b6cae-173c-461c-9195-4dce90bb6656-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-6krqs\" (UID: \"df6b6cae-173c-461c-9195-4dce90bb6656\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6krqs" Oct 30 22:08:09.196992 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.196895 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/df6b6cae-173c-461c-9195-4dce90bb6656-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-6krqs\" (UID: \"df6b6cae-173c-461c-9195-4dce90bb6656\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6krqs" Oct 30 22:08:09.196992 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.196963 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e129700d-0583-4316-bcee-431c57b0effa-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-xs8x9\" (UID: \"e129700d-0583-4316-bcee-431c57b0effa\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-xs8x9" Oct 30 22:08:09.197206 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.197128 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e129700d-0583-4316-bcee-431c57b0effa-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-xs8x9\" (UID: \"e129700d-0583-4316-bcee-431c57b0effa\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-xs8x9" Oct 30 22:08:09.197206 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.197160 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/df6b6cae-173c-461c-9195-4dce90bb6656-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-6krqs\" (UID: \"df6b6cae-173c-461c-9195-4dce90bb6656\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6krqs" Oct 30 22:08:09.197326 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.197259 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e129700d-0583-4316-bcee-431c57b0effa-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-xs8x9\" (UID: \"e129700d-0583-4316-bcee-431c57b0effa\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-xs8x9" Oct 30 22:08:09.197326 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.197284 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/df6b6cae-173c-461c-9195-4dce90bb6656-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-6krqs\" (UID: \"df6b6cae-173c-461c-9195-4dce90bb6656\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6krqs" Oct 30 22:08:09.197326 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.197309 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/df6b6cae-173c-461c-9195-4dce90bb6656-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-6krqs\" (UID: \"df6b6cae-173c-461c-9195-4dce90bb6656\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6krqs" Oct 30 22:08:09.205177 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.204897 2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-s7svn"] Oct 30 22:08:09.205308 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.205192 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Oct 30 22:08:09.207181 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.207157 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-6krqs"] Oct 30 22:08:09.207294 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.207281 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:09.235558 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.235521 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Oct 30 22:08:09.235725 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.235583 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Oct 30 22:08:09.235725 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.235649 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Oct 30 22:08:09.236050 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.236033 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-jctzf\"" Oct 30 22:08:09.236538 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.236520 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-t24hw\"" Oct 30 22:08:09.236647 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.236633 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Oct 30 22:08:09.236984 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.236969 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Oct 30 22:08:09.298547 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.298506 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/df6b6cae-173c-461c-9195-4dce90bb6656-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-6krqs\" (UID: \"df6b6cae-173c-461c-9195-4dce90bb6656\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6krqs" Oct 30 22:08:09.298713 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.298567 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e129700d-0583-4316-bcee-431c57b0effa-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-xs8x9\" (UID: \"e129700d-0583-4316-bcee-431c57b0effa\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-xs8x9" Oct 30 22:08:09.298713 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.298595 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68935e75-a4f0-4a54-9094-4f12b4a2dc39-metrics-client-ca\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:09.298713 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.298639 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/68935e75-a4f0-4a54-9094-4f12b4a2dc39-node-exporter-tls\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:09.298713 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.298670 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/df6b6cae-173c-461c-9195-4dce90bb6656-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-6krqs\" (UID: \"df6b6cae-173c-461c-9195-4dce90bb6656\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6krqs" Oct 30 22:08:09.298713 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.298696 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e129700d-0583-4316-bcee-431c57b0effa-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-xs8x9\" (UID: \"e129700d-0583-4316-bcee-431c57b0effa\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-xs8x9" Oct 30 22:08:09.298975 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.298722 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/68935e75-a4f0-4a54-9094-4f12b4a2dc39-node-exporter-textfile\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:09.298975 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.298810 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6fv5\" (UniqueName: \"kubernetes.io/projected/68935e75-a4f0-4a54-9094-4f12b4a2dc39-kube-api-access-z6fv5\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:09.298975 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.298879 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/68935e75-a4f0-4a54-9094-4f12b4a2dc39-node-exporter-wtmp\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:09.298975 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.298925 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/df6b6cae-173c-461c-9195-4dce90bb6656-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-6krqs\" (UID: \"df6b6cae-173c-461c-9195-4dce90bb6656\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6krqs" Oct 30 22:08:09.298975 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.298953 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/68935e75-a4f0-4a54-9094-4f12b4a2dc39-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:09.299213 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.298988 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/df6b6cae-173c-461c-9195-4dce90bb6656-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-6krqs\" (UID: \"df6b6cae-173c-461c-9195-4dce90bb6656\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6krqs" Oct 30 22:08:09.299213 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.299041 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zxs5\" (UniqueName: \"kubernetes.io/projected/e129700d-0583-4316-bcee-431c57b0effa-kube-api-access-5zxs5\") pod \"openshift-state-metrics-7bd7489957-xs8x9\" (UID: \"e129700d-0583-4316-bcee-431c57b0effa\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-xs8x9" Oct 30 22:08:09.299213 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.299068 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/df6b6cae-173c-461c-9195-4dce90bb6656-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-6krqs\" (UID: \"df6b6cae-173c-461c-9195-4dce90bb6656\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6krqs" Oct 30 22:08:09.299213 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.299096 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/68935e75-a4f0-4a54-9094-4f12b4a2dc39-node-exporter-accelerators-collector-config\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:09.299213 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.299124 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/68935e75-a4f0-4a54-9094-4f12b4a2dc39-sys\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:09.299442 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.299254 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45l5v\" (UniqueName: \"kubernetes.io/projected/df6b6cae-173c-461c-9195-4dce90bb6656-kube-api-access-45l5v\") pod \"kube-state-metrics-5dc8d7d89b-6krqs\" (UID: \"df6b6cae-173c-461c-9195-4dce90bb6656\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6krqs" Oct 30 22:08:09.299442 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.299292 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/68935e75-a4f0-4a54-9094-4f12b4a2dc39-root\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:09.299442 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.299435 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/df6b6cae-173c-461c-9195-4dce90bb6656-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-6krqs\" (UID: \"df6b6cae-173c-461c-9195-4dce90bb6656\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6krqs" Oct 30 22:08:09.299731 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.299706 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/df6b6cae-173c-461c-9195-4dce90bb6656-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-6krqs\" (UID: \"df6b6cae-173c-461c-9195-4dce90bb6656\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6krqs" Oct 30 22:08:09.299841 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.299823 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e129700d-0583-4316-bcee-431c57b0effa-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-xs8x9\" (UID: \"e129700d-0583-4316-bcee-431c57b0effa\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-xs8x9" Oct 30 22:08:09.300080 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.300056 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/df6b6cae-173c-461c-9195-4dce90bb6656-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-6krqs\" (UID: \"df6b6cae-173c-461c-9195-4dce90bb6656\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6krqs" Oct 30 22:08:09.300580 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.300555 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e129700d-0583-4316-bcee-431c57b0effa-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-xs8x9\" (UID: \"e129700d-0583-4316-bcee-431c57b0effa\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-xs8x9" Oct 30 22:08:09.301895 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.301871 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e129700d-0583-4316-bcee-431c57b0effa-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-xs8x9\" (UID: \"e129700d-0583-4316-bcee-431c57b0effa\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-xs8x9" Oct 30 22:08:09.301993 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.301875 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e129700d-0583-4316-bcee-431c57b0effa-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-xs8x9\" (UID: \"e129700d-0583-4316-bcee-431c57b0effa\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-xs8x9" Oct 30 22:08:09.302358 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.302333 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/df6b6cae-173c-461c-9195-4dce90bb6656-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-6krqs\" (UID: \"df6b6cae-173c-461c-9195-4dce90bb6656\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6krqs" Oct 30 22:08:09.303890 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.303867 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/df6b6cae-173c-461c-9195-4dce90bb6656-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-6krqs\" (UID: \"df6b6cae-173c-461c-9195-4dce90bb6656\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6krqs" Oct 30 22:08:09.323321 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.323288 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-45l5v\" (UniqueName: \"kubernetes.io/projected/df6b6cae-173c-461c-9195-4dce90bb6656-kube-api-access-45l5v\") pod \"kube-state-metrics-5dc8d7d89b-6krqs\" (UID: \"df6b6cae-173c-461c-9195-4dce90bb6656\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6krqs" Oct 30 22:08:09.328832 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.328615 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zxs5\" (UniqueName: \"kubernetes.io/projected/e129700d-0583-4316-bcee-431c57b0effa-kube-api-access-5zxs5\") pod \"openshift-state-metrics-7bd7489957-xs8x9\" (UID: \"e129700d-0583-4316-bcee-431c57b0effa\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-xs8x9" Oct 30 22:08:09.400394 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.400345 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/68935e75-a4f0-4a54-9094-4f12b4a2dc39-node-exporter-textfile\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:09.400536 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.400408 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6fv5\" (UniqueName: \"kubernetes.io/projected/68935e75-a4f0-4a54-9094-4f12b4a2dc39-kube-api-access-z6fv5\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:09.400536 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.400461 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/68935e75-a4f0-4a54-9094-4f12b4a2dc39-node-exporter-wtmp\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:09.400536 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.400490 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/68935e75-a4f0-4a54-9094-4f12b4a2dc39-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:09.400696 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.400545 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/68935e75-a4f0-4a54-9094-4f12b4a2dc39-node-exporter-accelerators-collector-config\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:09.400696 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.400570 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/68935e75-a4f0-4a54-9094-4f12b4a2dc39-sys\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:09.400696 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.400593 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/68935e75-a4f0-4a54-9094-4f12b4a2dc39-root\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:09.400696 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.400655 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68935e75-a4f0-4a54-9094-4f12b4a2dc39-metrics-client-ca\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:09.400848 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.400698 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/68935e75-a4f0-4a54-9094-4f12b4a2dc39-node-exporter-tls\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:09.401592 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.401240 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/68935e75-a4f0-4a54-9094-4f12b4a2dc39-sys\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:09.401789 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.401764 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/68935e75-a4f0-4a54-9094-4f12b4a2dc39-node-exporter-textfile\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:09.401870 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.401846 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/68935e75-a4f0-4a54-9094-4f12b4a2dc39-node-exporter-accelerators-collector-config\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:09.401950 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.401850 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/68935e75-a4f0-4a54-9094-4f12b4a2dc39-root\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:09.402005 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.401965 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/68935e75-a4f0-4a54-9094-4f12b4a2dc39-node-exporter-wtmp\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:09.402382 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.402359 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68935e75-a4f0-4a54-9094-4f12b4a2dc39-metrics-client-ca\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:09.403923 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.403898 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/68935e75-a4f0-4a54-9094-4f12b4a2dc39-node-exporter-tls\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:09.404475 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.404455 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/68935e75-a4f0-4a54-9094-4f12b4a2dc39-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:09.425083 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.425057 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-xs8x9" Oct 30 22:08:09.425719 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.425699 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6fv5\" (UniqueName: \"kubernetes.io/projected/68935e75-a4f0-4a54-9094-4f12b4a2dc39-kube-api-access-z6fv5\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:09.489286 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.489184 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6krqs" Oct 30 22:08:09.524456 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.524209 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:09.539613 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:08:09.539510 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68935e75_a4f0_4a54_9094_4f12b4a2dc39.slice/crio-62773096baa81d57351c85102abff61a5273833dd16c64c61a2dbbc4c56411dd WatchSource:0}: Error finding container 62773096baa81d57351c85102abff61a5273833dd16c64c61a2dbbc4c56411dd: Status 404 returned error can't find the container with id 62773096baa81d57351c85102abff61a5273833dd16c64c61a2dbbc4c56411dd Oct 30 22:08:09.736300 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.736275 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-xs8x9"] Oct 30 22:08:09.738125 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:08:09.738099 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode129700d_0583_4316_bcee_431c57b0effa.slice/crio-4caf4e2411d03431dff003062637fa6d121bfffb6fc2c67e46afdbad23627e6a WatchSource:0}: Error finding container 4caf4e2411d03431dff003062637fa6d121bfffb6fc2c67e46afdbad23627e6a: Status 404 returned error can't find the container with id 4caf4e2411d03431dff003062637fa6d121bfffb6fc2c67e46afdbad23627e6a Oct 30 22:08:09.816206 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.816180 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-6krqs"] Oct 30 22:08:09.906674 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.906633 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-xs8x9" event={"ID":"e129700d-0583-4316-bcee-431c57b0effa","Type":"ContainerStarted","Data":"1097f6b909fc7be7541543c31016e7a4d8027d42c32912b73fb4c300acfc46f1"} Oct 30 22:08:09.906762 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.906713 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-xs8x9" event={"ID":"e129700d-0583-4316-bcee-431c57b0effa","Type":"ContainerStarted","Data":"ba1146b0882206eefa04c23e5e537831d45cf0b34889e7fe47f56aa47db30708"} Oct 30 22:08:09.906762 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.906742 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-xs8x9" event={"ID":"e129700d-0583-4316-bcee-431c57b0effa","Type":"ContainerStarted","Data":"4caf4e2411d03431dff003062637fa6d121bfffb6fc2c67e46afdbad23627e6a"} Oct 30 22:08:09.909034 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.908977 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pn4kg" event={"ID":"5dfaf077-3637-4cc9-8681-466c4e2cc5f4","Type":"ContainerStarted","Data":"05e5621e19efe02e4977dd3dd366caefc62ed809601b86f80e1db0556f7cd048"} Oct 30 22:08:09.910701 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.910679 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s7svn" event={"ID":"68935e75-a4f0-4a54-9094-4f12b4a2dc39","Type":"ContainerStarted","Data":"62773096baa81d57351c85102abff61a5273833dd16c64c61a2dbbc4c56411dd"} Oct 30 22:08:09.912147 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:09.912127 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6krqs" event={"ID":"df6b6cae-173c-461c-9195-4dce90bb6656","Type":"ContainerStarted","Data":"340578bb6c446168bf7e528fda076049b3664bf8c0ce11cb85be81554dd5d793"} Oct 30 22:08:10.624489 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.624449 2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 30 22:08:10.628111 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.628090 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:10.646416 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.646389 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Oct 30 22:08:10.646538 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.646464 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Oct 30 22:08:10.646538 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.646532 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Oct 30 22:08:10.646702 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.646687 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Oct 30 22:08:10.646759 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.646750 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Oct 30 22:08:10.646864 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.646825 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Oct 30 22:08:10.656893 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.656873 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Oct 30 22:08:10.657016 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.656977 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-5s2rl\"" Oct 30 22:08:10.657016 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.657003 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Oct 30 22:08:10.669557 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.669533 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Oct 30 22:08:10.670973 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.670952 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 30 22:08:10.711822 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.711799 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:10.711898 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.711840 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-config-out\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:10.711898 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.711869 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:10.712017 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.711957 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:10.712017 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.712006 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpcht\" (UniqueName: \"kubernetes.io/projected/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-kube-api-access-bpcht\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:10.712162 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.712053 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:10.712162 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.712099 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-web-config\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:10.712284 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.712182 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:10.712284 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.712210 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:10.712383 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.712323 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:10.712434 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.712375 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:10.712434 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.712405 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:10.712529 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.712444 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-config-volume\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:10.813496 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.813466 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:10.813753 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.813506 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:10.813753 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.813537 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:10.813753 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.813563 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:10.813753 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.813579 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:10.813753 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.813601 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-config-volume\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:10.813753 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.813627 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:10.813753 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.813647 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-config-out\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:10.813753 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.813661 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:10.813753 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.813679 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:10.813753 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.813707 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpcht\" (UniqueName: \"kubernetes.io/projected/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-kube-api-access-bpcht\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:10.813753 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.813733 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:10.814058 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.813760 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-web-config\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:10.815246 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.814721 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:10.815246 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.815237 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:10.816744 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.816194 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:10.818903 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.818850 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-config-volume\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:10.821706 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.821191 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:10.821706 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.821664 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:10.823607 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.823415 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:10.823607 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.823508 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:10.823607 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.823573 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-config-out\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:10.823974 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.823934 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:10.825039 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.824136 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-web-config\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:10.826863 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.826837 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:10.838461 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.838418 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpcht\" (UniqueName: \"kubernetes.io/projected/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-kube-api-access-bpcht\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:10.918309 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.918273 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pl4x2" event={"ID":"c373c767-d743-4819-be7c-4c9c830eb9db","Type":"ContainerStarted","Data":"ac73a251fc5a355621e83549e60a9752f457d8a62f29bf596052c722a8a27e96"} Oct 30 22:08:10.918448 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.918373 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pl4x2" event={"ID":"c373c767-d743-4819-be7c-4c9c830eb9db","Type":"ContainerStarted","Data":"6b0eb66d7593fdc49b5a2d12f57bff914674185d9dbd41a83a2da8d8e3819c15"} Oct 30 22:08:10.919166 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.919142 2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-pl4x2" Oct 30 22:08:10.921908 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.921875 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pn4kg" event={"ID":"5dfaf077-3637-4cc9-8681-466c4e2cc5f4","Type":"ContainerStarted","Data":"145d11ddd3c4fe650baa8c44802c94ba8f7d90ad4d240d426eda3d3a3fa881f9"} Oct 30 22:08:10.924382 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.924349 2581 generic.go:356] "Generic (PLEG): container finished" podID="68935e75-a4f0-4a54-9094-4f12b4a2dc39" containerID="6814454a571673b27d352100071f0cc3eb1aa6bad6f3f89ace251007ca7e8b5c" exitCode=0 Oct 30 22:08:10.924492 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.924384 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s7svn" event={"ID":"68935e75-a4f0-4a54-9094-4f12b4a2dc39","Type":"ContainerDied","Data":"6814454a571673b27d352100071f0cc3eb1aa6bad6f3f89ace251007ca7e8b5c"} Oct 30 22:08:10.949362 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.949137 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:10.966250 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:10.965334 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-pl4x2" podStartSLOduration=1.9402630090000001 podStartE2EDuration="3.965317612s" totalImagesPullingTime="2.025054603s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:10.964446711 +0000 UTC m=+49.890549035" watchObservedRunningTime="2026-10-30 22:08:10.965317612 +0000 UTC m=+49.891419957" Oct 30 22:08:11.058083 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.057874 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-pn4kg" podStartSLOduration=1.5282267059999999 podStartE2EDuration="4.057854189s" totalImagesPullingTime="2.529627483s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:08:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:11.010670527 +0000 UTC m=+49.936772851" watchObservedRunningTime="2026-10-30 22:08:11.057854189 +0000 UTC m=+49.983956514" Oct 30 22:08:11.129611 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.129534 2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-6d4659d6dd-rb87s"] Oct 30 22:08:11.131624 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.131455 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:11.152890 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.152869 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Oct 30 22:08:11.153001 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.152951 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Oct 30 22:08:11.153001 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.152993 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Oct 30 22:08:11.153997 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.153976 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Oct 30 22:08:11.156801 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.156778 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Oct 30 22:08:11.156931 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.156876 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-d2gfd\"" Oct 30 22:08:11.157011 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.156987 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Oct 30 22:08:11.157731 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.157670 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d4659d6dd-rb87s"] Oct 30 22:08:11.224509 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.224485 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-oauth-serving-cert\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:11.224688 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.224547 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2kkt\" (UniqueName: \"kubernetes.io/projected/94283f0e-a719-4152-aea8-e219f7bb7eef-kube-api-access-k2kkt\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:11.224688 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.224608 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-console-config\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:11.224688 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.224631 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-trusted-ca-bundle\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:11.224688 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.224660 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-serving-cert\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:11.224688 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.224685 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-oauth-config\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:11.224955 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.224708 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-service-ca\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:11.224955 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.224742 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94283f0e-a719-4152-aea8-e219f7bb7eef-tmp\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:11.253024 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.252994 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Oct 30 22:08:11.253138 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.253033 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Oct 30 22:08:11.278265 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.278162 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 30 22:08:11.281823 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:08:11.281796 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46b06955_7c3e_4a10_b5cd_5c2f4c34859f.slice/crio-4e612a431c19e72aa48209c41de2f70414134eda7dbfd4fe026be056f9641119 WatchSource:0}: Error finding container 4e612a431c19e72aa48209c41de2f70414134eda7dbfd4fe026be056f9641119: Status 404 returned error can't find the container with id 4e612a431c19e72aa48209c41de2f70414134eda7dbfd4fe026be056f9641119 Oct 30 22:08:11.325610 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.325575 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2kkt\" (UniqueName: \"kubernetes.io/projected/94283f0e-a719-4152-aea8-e219f7bb7eef-kube-api-access-k2kkt\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:11.325743 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.325637 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-console-config\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:11.325743 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.325668 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-trusted-ca-bundle\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:11.325743 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.325700 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-serving-cert\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:11.325743 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.325717 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-oauth-config\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:11.325743 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.325737 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-service-ca\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:11.325999 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.325772 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94283f0e-a719-4152-aea8-e219f7bb7eef-tmp\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:11.325999 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.325801 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-oauth-serving-cert\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:11.326494 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.326462 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-oauth-serving-cert\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:11.327241 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.327205 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-console-config\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:11.327898 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.327877 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-trusted-ca-bundle\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:11.328550 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.328529 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94283f0e-a719-4152-aea8-e219f7bb7eef-tmp\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:11.328867 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.328806 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-service-ca\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:11.330595 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.330566 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-serving-cert\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:11.331030 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.331009 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-oauth-config\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:11.350006 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.349963 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2kkt\" (UniqueName: \"kubernetes.io/projected/94283f0e-a719-4152-aea8-e219f7bb7eef-kube-api-access-k2kkt\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:11.408841 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.408811 2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5ddf98bd44-xfj64"] Oct 30 22:08:11.412166 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.411489 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:11.427117 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.427093 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Oct 30 22:08:11.439014 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.438807 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5ddf98bd44-xfj64"] Oct 30 22:08:11.444616 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.444585 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:11.527285 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.526875 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1b444016-092c-472b-979a-4a7b16271e27-secret-grpc-tls\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:11.527285 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.526926 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1b444016-092c-472b-979a-4a7b16271e27-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:11.527285 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.526957 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b444016-092c-472b-979a-4a7b16271e27-metrics-client-ca\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:11.527285 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.527021 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9862\" (UniqueName: \"kubernetes.io/projected/1b444016-092c-472b-979a-4a7b16271e27-kube-api-access-j9862\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:11.527285 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.527066 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/1b444016-092c-472b-979a-4a7b16271e27-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:11.527285 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.527100 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1b444016-092c-472b-979a-4a7b16271e27-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:11.527285 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.527123 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/1b444016-092c-472b-979a-4a7b16271e27-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:11.527285 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.527157 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/1b444016-092c-472b-979a-4a7b16271e27-secret-thanos-querier-tls\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:11.527285 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.527183 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b444016-092c-472b-979a-4a7b16271e27-tmp\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:11.528251 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.528208 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Oct 30 22:08:11.528356 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.528341 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-pgqsk\"" Oct 30 22:08:11.528424 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.528407 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Oct 30 22:08:11.528516 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.528501 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Oct 30 22:08:11.528620 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.528606 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-vvmekr9k6f8j\"" Oct 30 22:08:11.528684 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.528675 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Oct 30 22:08:11.628846 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.628812 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/1b444016-092c-472b-979a-4a7b16271e27-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:11.628997 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.628870 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1b444016-092c-472b-979a-4a7b16271e27-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:11.628997 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.628901 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/1b444016-092c-472b-979a-4a7b16271e27-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:11.628997 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.628940 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/1b444016-092c-472b-979a-4a7b16271e27-secret-thanos-querier-tls\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:11.628997 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.628966 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b444016-092c-472b-979a-4a7b16271e27-tmp\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:11.629171 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.629042 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1b444016-092c-472b-979a-4a7b16271e27-secret-grpc-tls\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:11.629171 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.629075 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1b444016-092c-472b-979a-4a7b16271e27-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:11.629171 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.629099 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b444016-092c-472b-979a-4a7b16271e27-metrics-client-ca\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:11.629171 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.629159 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9862\" (UniqueName: \"kubernetes.io/projected/1b444016-092c-472b-979a-4a7b16271e27-kube-api-access-j9862\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:11.629770 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.629600 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b444016-092c-472b-979a-4a7b16271e27-tmp\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:11.630344 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.630318 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b444016-092c-472b-979a-4a7b16271e27-metrics-client-ca\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:11.632017 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.631996 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1b444016-092c-472b-979a-4a7b16271e27-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:11.632134 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.632113 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/1b444016-092c-472b-979a-4a7b16271e27-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:11.632247 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.632210 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1b444016-092c-472b-979a-4a7b16271e27-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:11.632733 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.632368 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/1b444016-092c-472b-979a-4a7b16271e27-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:11.632733 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.632697 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/1b444016-092c-472b-979a-4a7b16271e27-secret-thanos-querier-tls\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:11.634293 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.634272 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1b444016-092c-472b-979a-4a7b16271e27-secret-grpc-tls\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:11.653395 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.653373 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9862\" (UniqueName: \"kubernetes.io/projected/1b444016-092c-472b-979a-4a7b16271e27-kube-api-access-j9862\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:11.741507 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.741462 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:11.762117 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.762091 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d4659d6dd-rb87s"] Oct 30 22:08:11.764428 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:08:11.764403 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94283f0e_a719_4152_aea8_e219f7bb7eef.slice/crio-2320e8be4a7af79dbed9ac74a3043708a1694e6a3e6ecd5a21a60e433f61e3b5 WatchSource:0}: Error finding container 2320e8be4a7af79dbed9ac74a3043708a1694e6a3e6ecd5a21a60e433f61e3b5: Status 404 returned error can't find the container with id 2320e8be4a7af79dbed9ac74a3043708a1694e6a3e6ecd5a21a60e433f61e3b5 Oct 30 22:08:11.930996 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.930911 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-xs8x9" event={"ID":"e129700d-0583-4316-bcee-431c57b0effa","Type":"ContainerStarted","Data":"8f3876f3f0d360b7b69fd09c7997ebfb5fa60ace456e66c1aae09cf9f46635a0"} Oct 30 22:08:11.932450 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.932419 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d4659d6dd-rb87s" event={"ID":"94283f0e-a719-4152-aea8-e219f7bb7eef","Type":"ContainerStarted","Data":"2320e8be4a7af79dbed9ac74a3043708a1694e6a3e6ecd5a21a60e433f61e3b5"} Oct 30 22:08:11.933971 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.933945 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"46b06955-7c3e-4a10-b5cd-5c2f4c34859f","Type":"ContainerStarted","Data":"4e612a431c19e72aa48209c41de2f70414134eda7dbfd4fe026be056f9641119"} Oct 30 22:08:11.935862 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.935836 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9mkk7" event={"ID":"f64980b7-13c0-40c3-aa3f-277916f24518","Type":"ContainerStarted","Data":"11260bf1f6f6d4f528b0875b2dcf0c72d90578f5ddb1b3d018b66d2d0a7216ed"} Oct 30 22:08:11.938450 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.938428 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s7svn" event={"ID":"68935e75-a4f0-4a54-9094-4f12b4a2dc39","Type":"ContainerStarted","Data":"148cd41ba1f9ba2f60cf53a93ffa14b9f0dbf1c31a0ca415fff87d59eb062e32"} Oct 30 22:08:11.938555 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.938478 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s7svn" event={"ID":"68935e75-a4f0-4a54-9094-4f12b4a2dc39","Type":"ContainerStarted","Data":"22d58e5403bf1a78eddcce124eb9234ef8823820146d99d117da8e95266850f3"} Oct 30 22:08:11.941344 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.941318 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6krqs" event={"ID":"df6b6cae-173c-461c-9195-4dce90bb6656","Type":"ContainerStarted","Data":"f2961b47c841a21d298d3e68bbc3581431ca4f5a506d5331d4ce9398bba7a64c"} Oct 30 22:08:11.941445 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.941372 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6krqs" event={"ID":"df6b6cae-173c-461c-9195-4dce90bb6656","Type":"ContainerStarted","Data":"8573af9cd710551c7d85d3c9f9752d2a7ea2c4ef14f1eafd61b366e0242ed1f7"} Oct 30 22:08:11.941445 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.941399 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6krqs" event={"ID":"df6b6cae-173c-461c-9195-4dce90bb6656","Type":"ContainerStarted","Data":"6a2baec02a05325b34b409f39528da9971dbc14d9c4906926627fe940df743bd"} Oct 30 22:08:11.976101 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:11.976051 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-xs8x9" podStartSLOduration=1.714993542 podStartE2EDuration="2.97603637s" totalImagesPullingTime="1.261042828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:11.975170224 +0000 UTC m=+50.901272547" watchObservedRunningTime="2026-10-30 22:08:11.97603637 +0000 UTC m=+50.902138694" Oct 30 22:08:12.021628 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:12.021572 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-s7svn" podStartSLOduration=1.77448053 podStartE2EDuration="3.021557948s" totalImagesPullingTime="1.169715071s" totalInitContainerRuntime="77.362347ms" isStatefulPod=true podCreationTimestamp="2026-10-30 22:08:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:12.02098756 +0000 UTC m=+50.947089887" watchObservedRunningTime="2026-10-30 22:08:12.021557948 +0000 UTC m=+50.947660272" Oct 30 22:08:12.056747 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:12.056721 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5ddf98bd44-xfj64"] Oct 30 22:08:12.057608 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:08:12.057571 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b444016_092c_472b_979a_4a7b16271e27.slice/crio-a432ddbd05d471d3613077a1a33b72f9dc711e4b0edb4221802c3fff94c648b8 WatchSource:0}: Error finding container a432ddbd05d471d3613077a1a33b72f9dc711e4b0edb4221802c3fff94c648b8: Status 404 returned error can't find the container with id a432ddbd05d471d3613077a1a33b72f9dc711e4b0edb4221802c3fff94c648b8 Oct 30 22:08:12.063931 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:12.063889 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-9mkk7" podStartSLOduration=1.548629823 podStartE2EDuration="5.063877504s" totalImagesPullingTime="3.515247681s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:12.063810496 +0000 UTC m=+50.989912819" watchObservedRunningTime="2026-10-30 22:08:12.063877504 +0000 UTC m=+50.989979827" Oct 30 22:08:12.107117 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:12.107006 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6krqs" podStartSLOduration=1.5137500259999999 podStartE2EDuration="3.106987045s" totalImagesPullingTime="1.593237019s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:12.106618501 +0000 UTC m=+51.032720827" watchObservedRunningTime="2026-10-30 22:08:12.106987045 +0000 UTC m=+51.033089373" Oct 30 22:08:12.946586 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:12.946494 2581 generic.go:356] "Generic (PLEG): container finished" podID="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" containerID="e599f9a04f1e39f29f36cd26b16b488cf06bbe03c9228c4fda6d8f10e43c0b31" exitCode=0 Oct 30 22:08:12.947027 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:12.946585 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"46b06955-7c3e-4a10-b5cd-5c2f4c34859f","Type":"ContainerDied","Data":"e599f9a04f1e39f29f36cd26b16b488cf06bbe03c9228c4fda6d8f10e43c0b31"} Oct 30 22:08:12.949070 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:12.949045 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" event={"ID":"1b444016-092c-472b-979a-4a7b16271e27","Type":"ContainerStarted","Data":"a432ddbd05d471d3613077a1a33b72f9dc711e4b0edb4221802c3fff94c648b8"} Oct 30 22:08:13.715437 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:13.714327 2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-f89f5d58d-95dk2"] Oct 30 22:08:13.716645 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:13.716586 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-f89f5d58d-95dk2" Oct 30 22:08:13.732038 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:13.732017 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Oct 30 22:08:13.732137 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:13.732117 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Oct 30 22:08:13.732252 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:13.732235 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Oct 30 22:08:13.743827 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:13.743802 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Oct 30 22:08:13.743827 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:13.743827 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-jt6r2\"" Oct 30 22:08:13.743945 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:13.743916 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-buf3qpeaoqr6i\"" Oct 30 22:08:13.746858 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:13.746826 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-f89f5d58d-95dk2"] Oct 30 22:08:13.750714 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:13.750693 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9ac70a73-f7f0-4b3e-bf76-09869caefea0-secret-metrics-server-tls\") pod \"metrics-server-f89f5d58d-95dk2\" (UID: \"9ac70a73-f7f0-4b3e-bf76-09869caefea0\") " pod="openshift-monitoring/metrics-server-f89f5d58d-95dk2" Oct 30 22:08:13.750826 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:13.750750 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9ac70a73-f7f0-4b3e-bf76-09869caefea0-metrics-server-audit-profiles\") pod \"metrics-server-f89f5d58d-95dk2\" (UID: \"9ac70a73-f7f0-4b3e-bf76-09869caefea0\") " pod="openshift-monitoring/metrics-server-f89f5d58d-95dk2" Oct 30 22:08:13.750826 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:13.750807 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9ac70a73-f7f0-4b3e-bf76-09869caefea0-client-ca-bundle\") pod \"metrics-server-f89f5d58d-95dk2\" (UID: \"9ac70a73-f7f0-4b3e-bf76-09869caefea0\") " pod="openshift-monitoring/metrics-server-f89f5d58d-95dk2" Oct 30 22:08:13.750925 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:13.750841 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ac70a73-f7f0-4b3e-bf76-09869caefea0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-f89f5d58d-95dk2\" (UID: \"9ac70a73-f7f0-4b3e-bf76-09869caefea0\") " pod="openshift-monitoring/metrics-server-f89f5d58d-95dk2" Oct 30 22:08:13.750925 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:13.750873 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9ac70a73-f7f0-4b3e-bf76-09869caefea0-secret-metrics-server-client-certs\") pod \"metrics-server-f89f5d58d-95dk2\" (UID: \"9ac70a73-f7f0-4b3e-bf76-09869caefea0\") " pod="openshift-monitoring/metrics-server-f89f5d58d-95dk2" Oct 30 22:08:13.751026 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:13.750932 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9ac70a73-f7f0-4b3e-bf76-09869caefea0-audit-log\") pod \"metrics-server-f89f5d58d-95dk2\" (UID: \"9ac70a73-f7f0-4b3e-bf76-09869caefea0\") " pod="openshift-monitoring/metrics-server-f89f5d58d-95dk2" Oct 30 22:08:13.751026 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:13.750957 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfsr2\" (UniqueName: \"kubernetes.io/projected/9ac70a73-f7f0-4b3e-bf76-09869caefea0-kube-api-access-bfsr2\") pod \"metrics-server-f89f5d58d-95dk2\" (UID: \"9ac70a73-f7f0-4b3e-bf76-09869caefea0\") " pod="openshift-monitoring/metrics-server-f89f5d58d-95dk2" Oct 30 22:08:13.852368 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:13.852325 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9ac70a73-f7f0-4b3e-bf76-09869caefea0-metrics-server-audit-profiles\") pod \"metrics-server-f89f5d58d-95dk2\" (UID: \"9ac70a73-f7f0-4b3e-bf76-09869caefea0\") " pod="openshift-monitoring/metrics-server-f89f5d58d-95dk2" Oct 30 22:08:13.852534 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:13.852414 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9ac70a73-f7f0-4b3e-bf76-09869caefea0-client-ca-bundle\") pod \"metrics-server-f89f5d58d-95dk2\" (UID: \"9ac70a73-f7f0-4b3e-bf76-09869caefea0\") " pod="openshift-monitoring/metrics-server-f89f5d58d-95dk2" Oct 30 22:08:13.852534 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:13.852455 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ac70a73-f7f0-4b3e-bf76-09869caefea0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-f89f5d58d-95dk2\" (UID: \"9ac70a73-f7f0-4b3e-bf76-09869caefea0\") " pod="openshift-monitoring/metrics-server-f89f5d58d-95dk2" Oct 30 22:08:13.852534 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:13.852487 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9ac70a73-f7f0-4b3e-bf76-09869caefea0-secret-metrics-server-client-certs\") pod \"metrics-server-f89f5d58d-95dk2\" (UID: \"9ac70a73-f7f0-4b3e-bf76-09869caefea0\") " pod="openshift-monitoring/metrics-server-f89f5d58d-95dk2" Oct 30 22:08:13.852670 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:13.852544 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9ac70a73-f7f0-4b3e-bf76-09869caefea0-audit-log\") pod \"metrics-server-f89f5d58d-95dk2\" (UID: \"9ac70a73-f7f0-4b3e-bf76-09869caefea0\") " pod="openshift-monitoring/metrics-server-f89f5d58d-95dk2" Oct 30 22:08:13.852670 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:13.852570 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfsr2\" (UniqueName: \"kubernetes.io/projected/9ac70a73-f7f0-4b3e-bf76-09869caefea0-kube-api-access-bfsr2\") pod \"metrics-server-f89f5d58d-95dk2\" (UID: \"9ac70a73-f7f0-4b3e-bf76-09869caefea0\") " pod="openshift-monitoring/metrics-server-f89f5d58d-95dk2" Oct 30 22:08:13.852670 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:13.852640 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9ac70a73-f7f0-4b3e-bf76-09869caefea0-secret-metrics-server-tls\") pod \"metrics-server-f89f5d58d-95dk2\" (UID: \"9ac70a73-f7f0-4b3e-bf76-09869caefea0\") " pod="openshift-monitoring/metrics-server-f89f5d58d-95dk2" Oct 30 22:08:13.853608 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:13.853577 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9ac70a73-f7f0-4b3e-bf76-09869caefea0-audit-log\") pod \"metrics-server-f89f5d58d-95dk2\" (UID: \"9ac70a73-f7f0-4b3e-bf76-09869caefea0\") " pod="openshift-monitoring/metrics-server-f89f5d58d-95dk2" Oct 30 22:08:13.853940 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:13.853894 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ac70a73-f7f0-4b3e-bf76-09869caefea0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-f89f5d58d-95dk2\" (UID: \"9ac70a73-f7f0-4b3e-bf76-09869caefea0\") " pod="openshift-monitoring/metrics-server-f89f5d58d-95dk2" Oct 30 22:08:13.854231 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:13.854164 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9ac70a73-f7f0-4b3e-bf76-09869caefea0-metrics-server-audit-profiles\") pod \"metrics-server-f89f5d58d-95dk2\" (UID: \"9ac70a73-f7f0-4b3e-bf76-09869caefea0\") " pod="openshift-monitoring/metrics-server-f89f5d58d-95dk2" Oct 30 22:08:13.856424 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:13.856400 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9ac70a73-f7f0-4b3e-bf76-09869caefea0-secret-metrics-server-tls\") pod \"metrics-server-f89f5d58d-95dk2\" (UID: \"9ac70a73-f7f0-4b3e-bf76-09869caefea0\") " pod="openshift-monitoring/metrics-server-f89f5d58d-95dk2" Oct 30 22:08:13.856424 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:13.856420 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9ac70a73-f7f0-4b3e-bf76-09869caefea0-secret-metrics-server-client-certs\") pod \"metrics-server-f89f5d58d-95dk2\" (UID: \"9ac70a73-f7f0-4b3e-bf76-09869caefea0\") " pod="openshift-monitoring/metrics-server-f89f5d58d-95dk2" Oct 30 22:08:13.857439 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:13.857416 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9ac70a73-f7f0-4b3e-bf76-09869caefea0-client-ca-bundle\") pod \"metrics-server-f89f5d58d-95dk2\" (UID: \"9ac70a73-f7f0-4b3e-bf76-09869caefea0\") " pod="openshift-monitoring/metrics-server-f89f5d58d-95dk2" Oct 30 22:08:13.876820 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:13.876796 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfsr2\" (UniqueName: \"kubernetes.io/projected/9ac70a73-f7f0-4b3e-bf76-09869caefea0-kube-api-access-bfsr2\") pod \"metrics-server-f89f5d58d-95dk2\" (UID: \"9ac70a73-f7f0-4b3e-bf76-09869caefea0\") " pod="openshift-monitoring/metrics-server-f89f5d58d-95dk2" Oct 30 22:08:13.961416 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:13.961384 2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-29lx9"] Oct 30 22:08:13.963944 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:13.963925 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-29lx9" Oct 30 22:08:13.984516 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:13.984493 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Oct 30 22:08:13.986960 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:13.986900 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-29lx9"] Oct 30 22:08:13.987129 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:13.987108 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-r6g4b\"" Oct 30 22:08:14.030334 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.030033 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-f89f5d58d-95dk2" Oct 30 22:08:14.054498 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.054464 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9c0d0893-e78b-48f3-bbe8-9f7d0381f42e-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-29lx9\" (UID: \"9c0d0893-e78b-48f3-bbe8-9f7d0381f42e\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-29lx9" Oct 30 22:08:14.054620 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.054520 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxr72\" (UniqueName: \"kubernetes.io/projected/9c0d0893-e78b-48f3-bbe8-9f7d0381f42e-kube-api-access-hxr72\") pod \"monitoring-plugin-77f4f76594-29lx9\" (UID: \"9c0d0893-e78b-48f3-bbe8-9f7d0381f42e\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-29lx9" Oct 30 22:08:14.156283 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.155663 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxr72\" (UniqueName: \"kubernetes.io/projected/9c0d0893-e78b-48f3-bbe8-9f7d0381f42e-kube-api-access-hxr72\") pod \"monitoring-plugin-77f4f76594-29lx9\" (UID: \"9c0d0893-e78b-48f3-bbe8-9f7d0381f42e\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-29lx9" Oct 30 22:08:14.156283 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.155821 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9c0d0893-e78b-48f3-bbe8-9f7d0381f42e-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-29lx9\" (UID: \"9c0d0893-e78b-48f3-bbe8-9f7d0381f42e\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-29lx9" Oct 30 22:08:14.158787 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.158763 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9c0d0893-e78b-48f3-bbe8-9f7d0381f42e-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-29lx9\" (UID: \"9c0d0893-e78b-48f3-bbe8-9f7d0381f42e\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-29lx9" Oct 30 22:08:14.180822 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.180796 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxr72\" (UniqueName: \"kubernetes.io/projected/9c0d0893-e78b-48f3-bbe8-9f7d0381f42e-kube-api-access-hxr72\") pod \"monitoring-plugin-77f4f76594-29lx9\" (UID: \"9c0d0893-e78b-48f3-bbe8-9f7d0381f42e\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-29lx9" Oct 30 22:08:14.276927 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.276851 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-29lx9" Oct 30 22:08:14.356658 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.356563 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-f89f5d58d-95dk2"] Oct 30 22:08:14.360952 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:08:14.360898 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ac70a73_f7f0_4b3e_bf76_09869caefea0.slice/crio-1f0618e251474d6ffb7acd4a8504e72915215cd2c4d8e67c1e553cced27ae6d2 WatchSource:0}: Error finding container 1f0618e251474d6ffb7acd4a8504e72915215cd2c4d8e67c1e553cced27ae6d2: Status 404 returned error can't find the container with id 1f0618e251474d6ffb7acd4a8504e72915215cd2c4d8e67c1e553cced27ae6d2 Oct 30 22:08:14.472024 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.471989 2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-74754b6cd9-49r6n"] Oct 30 22:08:14.474761 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.474718 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" Oct 30 22:08:14.493073 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.493052 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Oct 30 22:08:14.505587 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.505562 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\"" Oct 30 22:08:14.514947 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.514924 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-74754b6cd9-49r6n"] Oct 30 22:08:14.551625 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.551603 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Oct 30 22:08:14.551625 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.551603 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Oct 30 22:08:14.551764 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.551663 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Oct 30 22:08:14.551764 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.551669 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Oct 30 22:08:14.551818 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.551768 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-gz6qb\"" Oct 30 22:08:14.559299 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.559277 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb3d3e48-51f2-4503-8026-79787ee98754-metrics-client-ca\") pod \"telemeter-client-74754b6cd9-49r6n\" (UID: \"fb3d3e48-51f2-4503-8026-79787ee98754\") " pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" Oct 30 22:08:14.559407 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.559328 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb3d3e48-51f2-4503-8026-79787ee98754-serving-certs-ca-bundle\") pod \"telemeter-client-74754b6cd9-49r6n\" (UID: \"fb3d3e48-51f2-4503-8026-79787ee98754\") " pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" Oct 30 22:08:14.559407 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.559363 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/fb3d3e48-51f2-4503-8026-79787ee98754-federate-client-tls\") pod \"telemeter-client-74754b6cd9-49r6n\" (UID: \"fb3d3e48-51f2-4503-8026-79787ee98754\") " pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" Oct 30 22:08:14.559407 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.559388 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtbtv\" (UniqueName: \"kubernetes.io/projected/fb3d3e48-51f2-4503-8026-79787ee98754-kube-api-access-jtbtv\") pod \"telemeter-client-74754b6cd9-49r6n\" (UID: \"fb3d3e48-51f2-4503-8026-79787ee98754\") " pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" Oct 30 22:08:14.559570 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.559422 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fb3d3e48-51f2-4503-8026-79787ee98754-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-74754b6cd9-49r6n\" (UID: \"fb3d3e48-51f2-4503-8026-79787ee98754\") " pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" Oct 30 22:08:14.559570 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.559497 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/fb3d3e48-51f2-4503-8026-79787ee98754-secret-telemeter-client\") pod \"telemeter-client-74754b6cd9-49r6n\" (UID: \"fb3d3e48-51f2-4503-8026-79787ee98754\") " pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" Oct 30 22:08:14.559570 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.559541 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/fb3d3e48-51f2-4503-8026-79787ee98754-telemeter-client-tls\") pod \"telemeter-client-74754b6cd9-49r6n\" (UID: \"fb3d3e48-51f2-4503-8026-79787ee98754\") " pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" Oct 30 22:08:14.559708 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.559590 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb3d3e48-51f2-4503-8026-79787ee98754-telemeter-trusted-ca-bundle\") pod \"telemeter-client-74754b6cd9-49r6n\" (UID: \"fb3d3e48-51f2-4503-8026-79787ee98754\") " pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" Oct 30 22:08:14.580367 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.580211 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-29lx9"] Oct 30 22:08:14.586672 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:08:14.586645 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c0d0893_e78b_48f3_bbe8_9f7d0381f42e.slice/crio-7062c9becc10226a3ca19a89d297a7269e2c51a6b6b7dca0f430f14dcd410149 WatchSource:0}: Error finding container 7062c9becc10226a3ca19a89d297a7269e2c51a6b6b7dca0f430f14dcd410149: Status 404 returned error can't find the container with id 7062c9becc10226a3ca19a89d297a7269e2c51a6b6b7dca0f430f14dcd410149 Oct 30 22:08:14.660835 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.660527 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb3d3e48-51f2-4503-8026-79787ee98754-telemeter-trusted-ca-bundle\") pod \"telemeter-client-74754b6cd9-49r6n\" (UID: \"fb3d3e48-51f2-4503-8026-79787ee98754\") " pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" Oct 30 22:08:14.660835 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.660586 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb3d3e48-51f2-4503-8026-79787ee98754-metrics-client-ca\") pod \"telemeter-client-74754b6cd9-49r6n\" (UID: \"fb3d3e48-51f2-4503-8026-79787ee98754\") " pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" Oct 30 22:08:14.660835 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.660629 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb3d3e48-51f2-4503-8026-79787ee98754-serving-certs-ca-bundle\") pod \"telemeter-client-74754b6cd9-49r6n\" (UID: \"fb3d3e48-51f2-4503-8026-79787ee98754\") " pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" Oct 30 22:08:14.660835 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.660658 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/fb3d3e48-51f2-4503-8026-79787ee98754-federate-client-tls\") pod \"telemeter-client-74754b6cd9-49r6n\" (UID: \"fb3d3e48-51f2-4503-8026-79787ee98754\") " pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" Oct 30 22:08:14.660835 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.660684 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtbtv\" (UniqueName: \"kubernetes.io/projected/fb3d3e48-51f2-4503-8026-79787ee98754-kube-api-access-jtbtv\") pod \"telemeter-client-74754b6cd9-49r6n\" (UID: \"fb3d3e48-51f2-4503-8026-79787ee98754\") " pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" Oct 30 22:08:14.660835 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.660718 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fb3d3e48-51f2-4503-8026-79787ee98754-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-74754b6cd9-49r6n\" (UID: \"fb3d3e48-51f2-4503-8026-79787ee98754\") " pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" Oct 30 22:08:14.660835 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.660793 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/fb3d3e48-51f2-4503-8026-79787ee98754-secret-telemeter-client\") pod \"telemeter-client-74754b6cd9-49r6n\" (UID: \"fb3d3e48-51f2-4503-8026-79787ee98754\") " pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" Oct 30 22:08:14.660835 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.660836 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/fb3d3e48-51f2-4503-8026-79787ee98754-telemeter-client-tls\") pod \"telemeter-client-74754b6cd9-49r6n\" (UID: \"fb3d3e48-51f2-4503-8026-79787ee98754\") " pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" Oct 30 22:08:14.665052 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.663896 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/fb3d3e48-51f2-4503-8026-79787ee98754-telemeter-client-tls\") pod \"telemeter-client-74754b6cd9-49r6n\" (UID: \"fb3d3e48-51f2-4503-8026-79787ee98754\") " pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" Oct 30 22:08:14.665052 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.665013 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb3d3e48-51f2-4503-8026-79787ee98754-metrics-client-ca\") pod \"telemeter-client-74754b6cd9-49r6n\" (UID: \"fb3d3e48-51f2-4503-8026-79787ee98754\") " pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" Oct 30 22:08:14.666384 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.665637 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb3d3e48-51f2-4503-8026-79787ee98754-serving-certs-ca-bundle\") pod \"telemeter-client-74754b6cd9-49r6n\" (UID: \"fb3d3e48-51f2-4503-8026-79787ee98754\") " pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" Oct 30 22:08:14.666384 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.666340 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb3d3e48-51f2-4503-8026-79787ee98754-telemeter-trusted-ca-bundle\") pod \"telemeter-client-74754b6cd9-49r6n\" (UID: \"fb3d3e48-51f2-4503-8026-79787ee98754\") " pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" Oct 30 22:08:14.666384 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.666374 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/fb3d3e48-51f2-4503-8026-79787ee98754-secret-telemeter-client\") pod \"telemeter-client-74754b6cd9-49r6n\" (UID: \"fb3d3e48-51f2-4503-8026-79787ee98754\") " pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" Oct 30 22:08:14.678510 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.678429 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/fb3d3e48-51f2-4503-8026-79787ee98754-federate-client-tls\") pod \"telemeter-client-74754b6cd9-49r6n\" (UID: \"fb3d3e48-51f2-4503-8026-79787ee98754\") " pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" Oct 30 22:08:14.679454 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.679410 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fb3d3e48-51f2-4503-8026-79787ee98754-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-74754b6cd9-49r6n\" (UID: \"fb3d3e48-51f2-4503-8026-79787ee98754\") " pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" Oct 30 22:08:14.688976 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.688931 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtbtv\" (UniqueName: \"kubernetes.io/projected/fb3d3e48-51f2-4503-8026-79787ee98754-kube-api-access-jtbtv\") pod \"telemeter-client-74754b6cd9-49r6n\" (UID: \"fb3d3e48-51f2-4503-8026-79787ee98754\") " pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" Oct 30 22:08:14.791355 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.791333 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" Oct 30 22:08:14.959290 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.959258 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-29lx9" event={"ID":"9c0d0893-e78b-48f3-bbe8-9f7d0381f42e","Type":"ContainerStarted","Data":"7062c9becc10226a3ca19a89d297a7269e2c51a6b6b7dca0f430f14dcd410149"} Oct 30 22:08:14.974259 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.971533 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"46b06955-7c3e-4a10-b5cd-5c2f4c34859f","Type":"ContainerStarted","Data":"b9059ad19c6204de4394dea117f2dedd2f6e8b4e62972234263b26a3289fd0db"} Oct 30 22:08:14.974259 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.971591 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"46b06955-7c3e-4a10-b5cd-5c2f4c34859f","Type":"ContainerStarted","Data":"5bccb4128b8010d35e1d2fedc4cf0f253a08a9cde42b580c48c64a4a93fe31fe"} Oct 30 22:08:14.974259 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.971624 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"46b06955-7c3e-4a10-b5cd-5c2f4c34859f","Type":"ContainerStarted","Data":"d0822fed9f2a2c5827ba836a0563738ad56a344412b85653ce9f04f1e25dbb2d"} Oct 30 22:08:14.974259 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.971648 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"46b06955-7c3e-4a10-b5cd-5c2f4c34859f","Type":"ContainerStarted","Data":"94c755efc0aedc9fb07704a28e0cee7df9af8edaa408e8451da92cb56f28f896"} Oct 30 22:08:14.974259 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.971671 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"46b06955-7c3e-4a10-b5cd-5c2f4c34859f","Type":"ContainerStarted","Data":"1a1dfe2f444e73e6026fc72da31a2f2e1a8bfc7f425e6552ed63ccc85b4e0171"} Oct 30 22:08:14.974866 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.974639 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" event={"ID":"1b444016-092c-472b-979a-4a7b16271e27","Type":"ContainerStarted","Data":"64924b3d6aea4464cd41ff57ad8147b3e4fc4e2cf75d79947eac57bd5a6e26cb"} Oct 30 22:08:14.974866 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.974689 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" event={"ID":"1b444016-092c-472b-979a-4a7b16271e27","Type":"ContainerStarted","Data":"e5d908032681929d908380025919744559981f2391fb1436a5e69a00eb6f64a2"} Oct 30 22:08:14.974866 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.974714 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" event={"ID":"1b444016-092c-472b-979a-4a7b16271e27","Type":"ContainerStarted","Data":"fb9839ddc838b98cc4ff41a9da0abbb2c469991777ccc639b573a540cfaf8338"} Oct 30 22:08:14.975920 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:14.975893 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-f89f5d58d-95dk2" event={"ID":"9ac70a73-f7f0-4b3e-bf76-09869caefea0","Type":"ContainerStarted","Data":"1f0618e251474d6ffb7acd4a8504e72915215cd2c4d8e67c1e553cced27ae6d2"} Oct 30 22:08:15.123358 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:15.123213 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-74754b6cd9-49r6n"] Oct 30 22:08:15.144358 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:08:15.144326 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb3d3e48_51f2_4503_8026_79787ee98754.slice/crio-cd91dfa17aa2f5dbb8bf2c7c3f923a026fe1805f799b2dd62fe87019e250f503 WatchSource:0}: Error finding container cd91dfa17aa2f5dbb8bf2c7c3f923a026fe1805f799b2dd62fe87019e250f503: Status 404 returned error can't find the container with id cd91dfa17aa2f5dbb8bf2c7c3f923a026fe1805f799b2dd62fe87019e250f503 Oct 30 22:08:15.982558 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:15.982518 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d4659d6dd-rb87s" event={"ID":"94283f0e-a719-4152-aea8-e219f7bb7eef","Type":"ContainerStarted","Data":"f2ae40eb21372803272ed17777f5660bced316438dd19549aaa7b0e05d4385dc"} Oct 30 22:08:15.987067 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:15.987039 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"46b06955-7c3e-4a10-b5cd-5c2f4c34859f","Type":"ContainerStarted","Data":"47d577d4ee928534952f088b26bc3c9b9746e81390f9b59c682ffdb0d7f0c885"} Oct 30 22:08:15.990774 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:15.990736 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" event={"ID":"1b444016-092c-472b-979a-4a7b16271e27","Type":"ContainerStarted","Data":"5af47744b79d7c6fac1b6d9d7da3a9e23b10ec4a1544e75061d17bc6cf28d6b1"} Oct 30 22:08:15.990871 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:15.990806 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" event={"ID":"1b444016-092c-472b-979a-4a7b16271e27","Type":"ContainerStarted","Data":"123c7d8442a751e33f6c8df9f02998586222091ea940a594617463807e2fffd3"} Oct 30 22:08:15.990871 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:15.990852 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" event={"ID":"1b444016-092c-472b-979a-4a7b16271e27","Type":"ContainerStarted","Data":"ff8583425dadafdb22cdcea89acefd48445b83f723b17b74c5a6083941bf0249"} Oct 30 22:08:15.990969 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:15.990898 2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:15.992159 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:15.992140 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" event={"ID":"fb3d3e48-51f2-4503-8026-79787ee98754","Type":"ContainerStarted","Data":"cd91dfa17aa2f5dbb8bf2c7c3f923a026fe1805f799b2dd62fe87019e250f503"} Oct 30 22:08:16.028916 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.028868 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-6d4659d6dd-rb87s" podStartSLOduration=1.740971592 podStartE2EDuration="5.028854504s" totalImagesPullingTime="3.287882912s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:16.02756287 +0000 UTC m=+54.953665195" watchObservedRunningTime="2026-10-30 22:08:16.028854504 +0000 UTC m=+54.954956827" Oct 30 22:08:16.082409 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.081572 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.848575071 podStartE2EDuration="6.081557738s" totalImagesPullingTime="3.219142007s" totalInitContainerRuntime="13.84066ms" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:10 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:16.080564844 +0000 UTC m=+55.006667207" watchObservedRunningTime="2026-10-30 22:08:16.081557738 +0000 UTC m=+55.007660062" Oct 30 22:08:16.134164 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.133941 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" podStartSLOduration=1.907896373 podStartE2EDuration="5.133919899s" totalImagesPullingTime="3.226023526s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:11 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:16.133255054 +0000 UTC m=+55.059357378" watchObservedRunningTime="2026-10-30 22:08:16.133919899 +0000 UTC m=+55.060022228" Oct 30 22:08:16.152623 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.152586 2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 30 22:08:16.156867 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.156845 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.172634 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.172612 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Oct 30 22:08:16.172701 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.172645 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Oct 30 22:08:16.172752 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.172701 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Oct 30 22:08:16.185923 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.185787 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Oct 30 22:08:16.185923 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.185852 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Oct 30 22:08:16.188114 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.188030 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 30 22:08:16.254118 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.254074 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Oct 30 22:08:16.254523 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.254341 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Oct 30 22:08:16.254523 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.254405 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Oct 30 22:08:16.254523 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.254419 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-anvv041vkp1fj\"" Oct 30 22:08:16.254523 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.254445 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Oct 30 22:08:16.254523 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.254333 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Oct 30 22:08:16.254523 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.254458 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Oct 30 22:08:16.254523 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.254485 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-p4pd4\"" Oct 30 22:08:16.254523 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.254497 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Oct 30 22:08:16.254937 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.254526 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Oct 30 22:08:16.255046 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.255031 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Oct 30 22:08:16.282919 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.282867 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.283051 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.282927 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.283051 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.282954 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.283051 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.282980 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-web-config\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.283051 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.283003 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.283415 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.283060 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.283415 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.283096 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.283415 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.283121 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsvp8\" (UniqueName: \"kubernetes.io/projected/3913a2c9-e88c-4f49-bc63-02b1fa543c38-kube-api-access-qsvp8\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.283415 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.283172 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.283415 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.283205 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.283415 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.283272 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-config\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.283415 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.283297 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.283415 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.283326 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.283415 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.283360 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.283415 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.283397 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.283906 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.283428 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.283906 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.283466 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3913a2c9-e88c-4f49-bc63-02b1fa543c38-config-out\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.283906 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.283495 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.283906 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.283522 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.283906 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.283572 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3913a2c9-e88c-4f49-bc63-02b1fa543c38-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.384616 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.384564 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.384778 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.384616 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.384845 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.384789 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-config\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.384845 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.384815 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.384955 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.384869 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.384955 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.384916 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.385055 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.384951 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.385055 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.384993 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.385055 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.385036 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3913a2c9-e88c-4f49-bc63-02b1fa543c38-config-out\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.385203 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.385060 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.385203 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.385084 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.385203 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.385130 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3913a2c9-e88c-4f49-bc63-02b1fa543c38-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.385203 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.385161 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.385423 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.385246 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.385423 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.385273 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.385423 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.385312 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-web-config\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.385423 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.385337 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.385608 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.385420 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.385608 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.385484 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.385608 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.385548 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsvp8\" (UniqueName: \"kubernetes.io/projected/3913a2c9-e88c-4f49-bc63-02b1fa543c38-kube-api-access-qsvp8\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.385608 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.385549 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.386088 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.386053 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.388752 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.386686 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.388752 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.387555 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.388752 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.387588 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.388752 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.387980 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-config\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.388752 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.388436 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.389891 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.389511 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.389891 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.389620 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3913a2c9-e88c-4f49-bc63-02b1fa543c38-config-out\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.389891 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.389831 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.390173 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.390152 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.390317 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.390295 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.391157 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.391097 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.391858 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.391830 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3913a2c9-e88c-4f49-bc63-02b1fa543c38-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.392038 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.392017 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.392098 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.392068 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-web-config\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.392588 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.392563 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.392675 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.392639 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.392784 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.392765 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.416001 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.415920 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsvp8\" (UniqueName: \"kubernetes.io/projected/3913a2c9-e88c-4f49-bc63-02b1fa543c38-kube-api-access-qsvp8\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.489007 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.488928 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:16.821407 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.821375 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 30 22:08:16.822355 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:08:16.822332 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3913a2c9_e88c_4f49_bc63_02b1fa543c38.slice/crio-2e1024a48494975a37cd70c11675061811824847d6e8d357b5b6836a0f397c37 WatchSource:0}: Error finding container 2e1024a48494975a37cd70c11675061811824847d6e8d357b5b6836a0f397c37: Status 404 returned error can't find the container with id 2e1024a48494975a37cd70c11675061811824847d6e8d357b5b6836a0f397c37 Oct 30 22:08:16.996627 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.996602 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-29lx9" event={"ID":"9c0d0893-e78b-48f3-bbe8-9f7d0381f42e","Type":"ContainerStarted","Data":"1c6849570621174debb37bd9abc2493bff316234f568263164ec9505dc4550a3"} Oct 30 22:08:16.996952 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.996774 2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-29lx9" Oct 30 22:08:16.998326 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.998298 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-f89f5d58d-95dk2" event={"ID":"9ac70a73-f7f0-4b3e-bf76-09869caefea0","Type":"ContainerStarted","Data":"4396d5a24be30ac35b356636eb8f60891f4b8d81bf9257763f18d73d050398ad"} Oct 30 22:08:16.999975 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:16.999954 2581 generic.go:356] "Generic (PLEG): container finished" podID="3913a2c9-e88c-4f49-bc63-02b1fa543c38" containerID="8b9b1cb2d4b58b1640e7b7ffa96201c957c0f209ed4cbb0101202683c3db5f62" exitCode=0 Oct 30 22:08:17.000112 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:17.000089 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3913a2c9-e88c-4f49-bc63-02b1fa543c38","Type":"ContainerDied","Data":"8b9b1cb2d4b58b1640e7b7ffa96201c957c0f209ed4cbb0101202683c3db5f62"} Oct 30 22:08:17.000179 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:17.000162 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3913a2c9-e88c-4f49-bc63-02b1fa543c38","Type":"ContainerStarted","Data":"2e1024a48494975a37cd70c11675061811824847d6e8d357b5b6836a0f397c37"} Oct 30 22:08:17.002122 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:17.002104 2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-29lx9" Oct 30 22:08:17.038693 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:17.038641 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-77f4f76594-29lx9" podStartSLOduration=2.457021145 podStartE2EDuration="4.038621684s" totalImagesPullingTime="1.581600539s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:17.037634198 +0000 UTC m=+55.963736523" watchObservedRunningTime="2026-10-30 22:08:17.038621684 +0000 UTC m=+55.964724009" Oct 30 22:08:17.131888 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:17.131829 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-f89f5d58d-95dk2" podStartSLOduration=2.328612077 podStartE2EDuration="4.131809495s" totalImagesPullingTime="1.803197418s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:17.131531989 +0000 UTC m=+56.057634306" watchObservedRunningTime="2026-10-30 22:08:17.131809495 +0000 UTC m=+56.057911820" Oct 30 22:08:17.409894 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:17.409871 2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:18.008100 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:18.007658 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" event={"ID":"fb3d3e48-51f2-4503-8026-79787ee98754","Type":"ContainerStarted","Data":"6dfc1ca64658de032b986d23a991c46f84c4a6cdaf4ca1f1676f336afc023ccb"} Oct 30 22:08:18.008100 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:18.007727 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" event={"ID":"fb3d3e48-51f2-4503-8026-79787ee98754","Type":"ContainerStarted","Data":"b2e44fabfbbdd69c2329e16935b4366453d35eecbfede0f75b42d027d51df974"} Oct 30 22:08:18.008100 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:18.007747 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" event={"ID":"fb3d3e48-51f2-4503-8026-79787ee98754","Type":"ContainerStarted","Data":"d3af99b09ddb885a4ec332de3997d44160d8b4e718826294912d52ff8718eb33"} Oct 30 22:08:18.055428 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:18.055370 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" podStartSLOduration=2.255966184 podStartE2EDuration="4.055351808s" totalImagesPullingTime="1.799385624s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:18.053846867 +0000 UTC m=+56.979949190" watchObservedRunningTime="2026-10-30 22:08:18.055351808 +0000 UTC m=+56.981454133" Oct 30 22:08:19.955442 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:19.955413 2581 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d4659d6dd-rb87s"] Oct 30 22:08:20.023252 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:20.020452 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3913a2c9-e88c-4f49-bc63-02b1fa543c38","Type":"ContainerStarted","Data":"6df39d2515c9a702cf3a1d64f1cc0a8a9e056bbb0fa98cf0ebaad9dad3096a05"} Oct 30 22:08:20.023252 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:20.020534 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3913a2c9-e88c-4f49-bc63-02b1fa543c38","Type":"ContainerStarted","Data":"36bd1a5bed3b82469994cb7d98f25114091efa66af5fe091e3037500ac0d995b"} Oct 30 22:08:21.027176 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:21.027140 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3913a2c9-e88c-4f49-bc63-02b1fa543c38","Type":"ContainerStarted","Data":"b2e637f872238fab5e1353aa253e2d1ec3b076550b69a82adb4d137d1af394d4"} Oct 30 22:08:21.027706 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:21.027208 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3913a2c9-e88c-4f49-bc63-02b1fa543c38","Type":"ContainerStarted","Data":"e65b0800081ec131f68d9c0efae55c42ac26fdd495967642fa18f904d0840737"} Oct 30 22:08:21.027706 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:21.027251 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3913a2c9-e88c-4f49-bc63-02b1fa543c38","Type":"ContainerStarted","Data":"911a2c5672e542f3af43a336f66776e42858c381f94b8a1fe92b1ca4b805c5f5"} Oct 30 22:08:21.027706 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:21.027287 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3913a2c9-e88c-4f49-bc63-02b1fa543c38","Type":"ContainerStarted","Data":"056a499f0455153c6c6409a71b6292bf1c957cd5d23143a835b0e63819b58859"} Oct 30 22:08:21.094951 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:21.094903 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.196191234 podStartE2EDuration="5.094889091s" totalImagesPullingTime="2.883399447s" totalInitContainerRuntime="15.29841ms" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:21.092936872 +0000 UTC m=+60.019039196" watchObservedRunningTime="2026-10-30 22:08:21.094889091 +0000 UTC m=+60.020991459" Oct 30 22:08:21.444740 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:21.444711 2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:21.489882 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:21.489852 2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:21.952042 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:21.952015 2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-pl4x2" Oct 30 22:08:22.006547 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:22.006519 2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:26.377388 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:26.377348 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65c2ef45-7211-4828-86bc-c9d7af39ec66-metrics-certs\") pod \"network-metrics-daemon-l7kkv\" (UID: \"65c2ef45-7211-4828-86bc-c9d7af39ec66\") " pod="openshift-multus/network-metrics-daemon-l7kkv" Oct 30 22:08:26.377819 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:26.377449 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cd46f3e8-c752-4eed-a65d-07e060bf32b4-original-pull-secret\") pod \"global-pull-secret-syncer-h9v2l\" (UID: \"cd46f3e8-c752-4eed-a65d-07e060bf32b4\") " pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:08:26.402902 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:26.402879 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Oct 30 22:08:26.402902 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:26.402892 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Oct 30 22:08:26.410201 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:26.410174 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65c2ef45-7211-4828-86bc-c9d7af39ec66-metrics-certs\") pod \"network-metrics-daemon-l7kkv\" (UID: \"65c2ef45-7211-4828-86bc-c9d7af39ec66\") " pod="openshift-multus/network-metrics-daemon-l7kkv" Oct 30 22:08:26.410307 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:26.410274 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cd46f3e8-c752-4eed-a65d-07e060bf32b4-original-pull-secret\") pod \"global-pull-secret-syncer-h9v2l\" (UID: \"cd46f3e8-c752-4eed-a65d-07e060bf32b4\") " pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:08:26.478786 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:26.478747 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gvkg\" (UniqueName: \"kubernetes.io/projected/3155dfa9-0dc2-4256-9e16-490b3721702d-kube-api-access-2gvkg\") pod \"network-check-target-knzz5\" (UID: \"3155dfa9-0dc2-4256-9e16-490b3721702d\") " pod="openshift-network-diagnostics/network-check-target-knzz5" Oct 30 22:08:26.495962 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:26.495940 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Oct 30 22:08:26.512716 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:26.512697 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Oct 30 22:08:26.521621 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:26.521601 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gvkg\" (UniqueName: \"kubernetes.io/projected/3155dfa9-0dc2-4256-9e16-490b3721702d-kube-api-access-2gvkg\") pod \"network-check-target-knzz5\" (UID: \"3155dfa9-0dc2-4256-9e16-490b3721702d\") " pod="openshift-network-diagnostics/network-check-target-knzz5" Oct 30 22:08:26.590322 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:26.590300 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5nx2r\"" Oct 30 22:08:26.596463 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:26.596447 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-hmbsc\"" Oct 30 22:08:26.597466 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:26.597454 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7kkv" Oct 30 22:08:26.602806 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:26.602780 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-gq78g\"" Oct 30 22:08:26.603751 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:26.603738 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-knzz5" Oct 30 22:08:26.609796 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:26.609779 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:08:26.930589 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:26.930558 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-knzz5"] Oct 30 22:08:26.931147 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:08:26.931125 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3155dfa9_0dc2_4256_9e16_490b3721702d.slice/crio-f70f76b15ecaf9d5fff82369559697fc0109804c1f08a91ec6e029cb3acea159 WatchSource:0}: Error finding container f70f76b15ecaf9d5fff82369559697fc0109804c1f08a91ec6e029cb3acea159: Status 404 returned error can't find the container with id f70f76b15ecaf9d5fff82369559697fc0109804c1f08a91ec6e029cb3acea159 Oct 30 22:08:26.933839 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:08:26.933816 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd46f3e8_c752_4eed_a65d_07e060bf32b4.slice/crio-d71deada5e7ff6a92740256adf5084afe01473c75b9427829a90d8b0c608cd3a WatchSource:0}: Error finding container d71deada5e7ff6a92740256adf5084afe01473c75b9427829a90d8b0c608cd3a: Status 404 returned error can't find the container with id d71deada5e7ff6a92740256adf5084afe01473c75b9427829a90d8b0c608cd3a Oct 30 22:08:26.934195 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:26.934173 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-l7kkv"] Oct 30 22:08:26.934685 ip-10-0-142-24 kubenswrapper[2581]: W1030 22:08:26.934532 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65c2ef45_7211_4828_86bc_c9d7af39ec66.slice/crio-1b3456bb2efcc685f0f73cec86542ed7117b7502a4cba412c802ad4c9d644ed6 WatchSource:0}: Error finding container 1b3456bb2efcc685f0f73cec86542ed7117b7502a4cba412c802ad4c9d644ed6: Status 404 returned error can't find the container with id 1b3456bb2efcc685f0f73cec86542ed7117b7502a4cba412c802ad4c9d644ed6 Oct 30 22:08:26.941933 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:26.941912 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h9v2l"] Oct 30 22:08:27.050194 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:27.050155 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l7kkv" event={"ID":"65c2ef45-7211-4828-86bc-c9d7af39ec66","Type":"ContainerStarted","Data":"1b3456bb2efcc685f0f73cec86542ed7117b7502a4cba412c802ad4c9d644ed6"} Oct 30 22:08:27.051389 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:27.051369 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-h9v2l" event={"ID":"cd46f3e8-c752-4eed-a65d-07e060bf32b4","Type":"ContainerStarted","Data":"d71deada5e7ff6a92740256adf5084afe01473c75b9427829a90d8b0c608cd3a"} Oct 30 22:08:27.052509 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:27.052492 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-knzz5" event={"ID":"3155dfa9-0dc2-4256-9e16-490b3721702d","Type":"ContainerStarted","Data":"f70f76b15ecaf9d5fff82369559697fc0109804c1f08a91ec6e029cb3acea159"} Oct 30 22:08:30.070132 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:30.070090 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l7kkv" event={"ID":"65c2ef45-7211-4828-86bc-c9d7af39ec66","Type":"ContainerStarted","Data":"b89b76fb94422c0673811af4e4d956d6ef9f325dbfa7f540ee5111cd24e9ac56"} Oct 30 22:08:30.070567 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:30.070164 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l7kkv" event={"ID":"65c2ef45-7211-4828-86bc-c9d7af39ec66","Type":"ContainerStarted","Data":"6e81e7aa44f2d1277b98bd7f47b513fe451475ab4405998097115f0df0901974"} Oct 30 22:08:30.113148 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:30.113087 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-l7kkv" podStartSLOduration=65.981817647 podStartE2EDuration="1m8.113072598s" totalImagesPullingTime="2.131254951s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:07:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:30.111599666 +0000 UTC m=+69.037701993" watchObservedRunningTime="2026-10-30 22:08:30.113072598 +0000 UTC m=+69.039174921" Oct 30 22:08:31.076688 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:31.076651 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-knzz5" event={"ID":"3155dfa9-0dc2-4256-9e16-490b3721702d","Type":"ContainerStarted","Data":"da4e8cc337d8d25784078bf7319434640b5ad5f05477b6993671cee94ea304c1"} Oct 30 22:08:31.077147 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:31.077121 2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-knzz5" Oct 30 22:08:31.117494 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:31.117431 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-knzz5" podStartSLOduration=66.874992503 podStartE2EDuration="1m10.117411978s" totalImagesPullingTime="3.242419475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:07:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:31.117023788 +0000 UTC m=+70.043126114" watchObservedRunningTime="2026-10-30 22:08:31.117411978 +0000 UTC m=+70.043514303" Oct 30 22:08:31.980902 ip-10-0-142-24 kubenswrapper[2581]: I1030 22:08:31.980826 2581 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 22:08:31.981003 ip-10-0-142-24 systemd[1]: Stopping Kubernetes Kubelet... Oct 30 22:08:31.990667 ip-10-0-142-24 systemd[1]: kubelet.service: Deactivated successfully. Oct 30 22:08:31.990831 ip-10-0-142-24 systemd[1]: Stopped Kubernetes Kubelet. Oct 30 22:08:31.990883 ip-10-0-142-24 systemd[1]: kubelet.service: Consumed 4.519s CPU time. Oct 30 22:08:32.004826 ip-10-0-142-24 systemd[1]: Starting Kubernetes Kubelet... Oct 30 22:08:32.068295 ip-10-0-142-24 kubenswrapper[5949]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 22:08:32.068295 ip-10-0-142-24 kubenswrapper[5949]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Oct 30 22:08:32.068295 ip-10-0-142-24 kubenswrapper[5949]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 22:08:32.068295 ip-10-0-142-24 kubenswrapper[5949]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 22:08:32.070608 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070523 5949 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 22:08:32.070608 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070603 5949 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 22:08:32.070608 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070609 5949 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 22:08:32.070608 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070613 5949 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 22:08:32.070871 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070617 5949 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 22:08:32.070871 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070622 5949 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 22:08:32.070871 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070626 5949 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 22:08:32.070871 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070630 5949 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 22:08:32.070871 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070634 5949 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 22:08:32.070871 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070638 5949 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 22:08:32.070871 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070642 5949 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 22:08:32.070871 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070646 5949 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 22:08:32.070871 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070650 5949 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 22:08:32.070871 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070663 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 22:08:32.070871 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070667 5949 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 22:08:32.070871 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070671 5949 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 22:08:32.070871 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070676 5949 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 22:08:32.070871 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070680 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 22:08:32.070871 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070684 5949 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 22:08:32.070871 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070688 5949 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 22:08:32.070871 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070691 5949 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 22:08:32.070871 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070695 5949 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 22:08:32.070871 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070699 5949 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 22:08:32.070871 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070703 5949 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 22:08:32.071789 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070707 5949 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 22:08:32.071789 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070711 5949 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 22:08:32.071789 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070717 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 22:08:32.071789 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070721 5949 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 22:08:32.071789 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070726 5949 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 22:08:32.071789 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070730 5949 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 22:08:32.071789 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070734 5949 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 22:08:32.071789 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070742 5949 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 22:08:32.071789 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070750 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 22:08:32.071789 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070755 5949 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 22:08:32.071789 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070760 5949 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 22:08:32.071789 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070765 5949 feature_gate.go:386] unrecognized feature gate: Example Oct 30 22:08:32.071789 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070770 5949 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 22:08:32.071789 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070774 5949 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 22:08:32.071789 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070780 5949 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 22:08:32.071789 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070785 5949 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 22:08:32.071789 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070790 5949 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 22:08:32.071789 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070794 5949 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 22:08:32.071789 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070799 5949 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 22:08:32.071789 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070803 5949 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 22:08:32.072722 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070807 5949 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 22:08:32.072722 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070812 5949 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 22:08:32.072722 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070817 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 22:08:32.072722 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070821 5949 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 22:08:32.072722 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070825 5949 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 22:08:32.072722 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070830 5949 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 22:08:32.072722 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070835 5949 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 22:08:32.072722 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070840 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 22:08:32.072722 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070845 5949 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 22:08:32.072722 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070849 5949 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 22:08:32.072722 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070853 5949 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 22:08:32.072722 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070858 5949 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 22:08:32.072722 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070862 5949 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 22:08:32.072722 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070866 5949 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 22:08:32.072722 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070872 5949 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 22:08:32.072722 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070876 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 22:08:32.072722 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070882 5949 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 22:08:32.072722 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070886 5949 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 22:08:32.072722 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070891 5949 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 22:08:32.072722 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070895 5949 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 22:08:32.073621 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070899 5949 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 22:08:32.073621 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070903 5949 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 22:08:32.073621 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070907 5949 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 22:08:32.073621 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070912 5949 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 22:08:32.073621 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070917 5949 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 22:08:32.073621 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070921 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 22:08:32.073621 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070925 5949 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 22:08:32.073621 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070930 5949 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 22:08:32.073621 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070934 5949 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 22:08:32.073621 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070938 5949 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 22:08:32.073621 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070943 5949 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 22:08:32.073621 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070947 5949 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 22:08:32.073621 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070951 5949 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 22:08:32.073621 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070956 5949 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 22:08:32.073621 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070961 5949 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 22:08:32.073621 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070966 5949 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 22:08:32.073621 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070970 5949 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 22:08:32.073621 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070975 5949 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 22:08:32.073621 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070979 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 22:08:32.074482 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070984 5949 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 22:08:32.074482 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070988 5949 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 22:08:32.074482 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070992 5949 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 22:08:32.074482 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.070997 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 22:08:32.074482 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.071001 5949 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 22:08:32.074482 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.071006 5949 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 22:08:32.074482 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.071011 5949 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 22:08:32.074482 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.071015 5949 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 22:08:32.074482 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.071020 5949 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 22:08:32.074482 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.071024 5949 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 22:08:32.074482 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.071029 5949 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 22:08:32.074482 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.071036 5949 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 22:08:32.074482 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.071041 5949 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 22:08:32.074482 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.071048 5949 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 22:08:32.074482 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.071055 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 22:08:32.074482 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.071059 5949 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 22:08:32.074482 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.071065 5949 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 22:08:32.074482 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.071069 5949 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 22:08:32.074482 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.071073 5949 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 22:08:32.075428 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.071077 5949 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 22:08:32.075428 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.071082 5949 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 22:08:32.075428 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.071086 5949 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 22:08:32.075428 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.071090 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 22:08:32.075428 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072378 5949 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 22:08:32.075428 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072389 5949 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 22:08:32.075428 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072394 5949 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 22:08:32.075428 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072399 5949 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 22:08:32.075428 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072403 5949 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 22:08:32.075428 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072408 5949 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 22:08:32.075428 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072414 5949 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 22:08:32.075428 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072418 5949 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 22:08:32.075428 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072423 5949 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 22:08:32.075428 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072427 5949 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 22:08:32.075428 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072432 5949 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 22:08:32.075428 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072436 5949 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 22:08:32.075428 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072441 5949 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 22:08:32.075428 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072445 5949 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 22:08:32.075428 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072450 5949 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 22:08:32.075428 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072455 5949 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 22:08:32.075428 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072459 5949 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 22:08:32.076403 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072463 5949 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 22:08:32.076403 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072468 5949 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 22:08:32.076403 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072473 5949 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 22:08:32.076403 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072478 5949 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 22:08:32.076403 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072482 5949 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 22:08:32.076403 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072487 5949 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 22:08:32.076403 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072491 5949 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 22:08:32.076403 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072496 5949 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 22:08:32.076403 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072501 5949 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 22:08:32.076403 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072508 5949 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 22:08:32.076403 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072512 5949 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 22:08:32.076403 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072516 5949 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 22:08:32.076403 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072520 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 22:08:32.076403 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072525 5949 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 22:08:32.076403 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072529 5949 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 22:08:32.076403 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072534 5949 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 22:08:32.076403 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072538 5949 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 22:08:32.076403 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072542 5949 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 22:08:32.076403 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072547 5949 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 22:08:32.077268 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072551 5949 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 22:08:32.077268 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072555 5949 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 22:08:32.077268 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072560 5949 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 22:08:32.077268 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072565 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 22:08:32.077268 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072569 5949 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 22:08:32.077268 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072573 5949 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 22:08:32.077268 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072578 5949 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 22:08:32.077268 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072582 5949 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 22:08:32.077268 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072587 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 22:08:32.077268 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072591 5949 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 22:08:32.077268 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072595 5949 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 22:08:32.077268 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072600 5949 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 22:08:32.077268 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072604 5949 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 22:08:32.077268 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072615 5949 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 22:08:32.077268 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072620 5949 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 22:08:32.077268 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072624 5949 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 22:08:32.077268 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072628 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 22:08:32.077268 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072632 5949 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 22:08:32.077268 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072637 5949 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 22:08:32.077268 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072641 5949 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 22:08:32.077882 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072646 5949 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 22:08:32.077882 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072650 5949 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 22:08:32.077882 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072656 5949 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 22:08:32.077882 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072660 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 22:08:32.077882 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072666 5949 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 22:08:32.077882 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072671 5949 feature_gate.go:386] unrecognized feature gate: Example Oct 30 22:08:32.077882 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072676 5949 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 22:08:32.077882 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072680 5949 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 22:08:32.077882 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072685 5949 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 22:08:32.077882 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072689 5949 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 22:08:32.077882 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072694 5949 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 22:08:32.077882 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072700 5949 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 22:08:32.077882 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072707 5949 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 22:08:32.077882 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072735 5949 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 22:08:32.077882 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072741 5949 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 22:08:32.077882 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072748 5949 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 22:08:32.077882 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072753 5949 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 22:08:32.077882 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072758 5949 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 22:08:32.077882 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072764 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 22:08:32.078430 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072769 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 22:08:32.078430 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072774 5949 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 22:08:32.078430 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072778 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 22:08:32.078430 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072783 5949 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 22:08:32.078430 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072787 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 22:08:32.078430 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072791 5949 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 22:08:32.078430 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072796 5949 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 22:08:32.078430 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072800 5949 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 22:08:32.078430 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072804 5949 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 22:08:32.078430 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072809 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 22:08:32.078430 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072813 5949 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 22:08:32.078430 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072817 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 22:08:32.078430 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072822 5949 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 22:08:32.078430 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072826 5949 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 22:08:32.078430 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072830 5949 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 22:08:32.078430 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072835 5949 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 22:08:32.078430 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072839 5949 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 22:08:32.078430 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072843 5949 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 22:08:32.078430 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072848 5949 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 22:08:32.078945 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072852 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 22:08:32.078945 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072856 5949 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 22:08:32.078945 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072860 5949 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 22:08:32.078945 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072864 5949 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 22:08:32.078945 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072869 5949 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 22:08:32.078945 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072873 5949 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 22:08:32.078945 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072878 5949 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 22:08:32.078945 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072882 5949 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 22:08:32.078945 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072887 5949 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 22:08:32.078945 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072891 5949 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 22:08:32.078945 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072895 5949 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 22:08:32.078945 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.072901 5949 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 22:08:32.078945 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074120 5949 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 22:08:32.078945 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074130 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 22:08:32.078945 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074135 5949 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 22:08:32.078945 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074140 5949 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 22:08:32.078945 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074145 5949 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 22:08:32.078945 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074149 5949 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 22:08:32.078945 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074153 5949 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 22:08:32.079478 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074158 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 22:08:32.079478 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074162 5949 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 22:08:32.079478 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074167 5949 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 22:08:32.079478 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074171 5949 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 22:08:32.079478 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074176 5949 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 22:08:32.079478 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074180 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 22:08:32.079478 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074184 5949 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 22:08:32.079478 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074189 5949 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 22:08:32.079478 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074193 5949 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 22:08:32.079478 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074198 5949 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 22:08:32.079478 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074203 5949 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 22:08:32.079478 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074207 5949 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 22:08:32.079478 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074212 5949 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 22:08:32.079478 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074235 5949 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 22:08:32.079478 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074243 5949 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 22:08:32.079478 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074249 5949 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 22:08:32.079478 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074253 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 22:08:32.079478 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074258 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 22:08:32.079478 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074262 5949 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 22:08:32.079952 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074266 5949 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 22:08:32.079952 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074273 5949 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 22:08:32.079952 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074278 5949 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 22:08:32.079952 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074283 5949 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 22:08:32.079952 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074287 5949 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 22:08:32.079952 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074293 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 22:08:32.079952 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074297 5949 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 22:08:32.079952 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074301 5949 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 22:08:32.079952 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074305 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 22:08:32.079952 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074309 5949 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 22:08:32.079952 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074314 5949 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 22:08:32.079952 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074319 5949 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 22:08:32.079952 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074323 5949 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 22:08:32.079952 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074327 5949 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 22:08:32.079952 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074332 5949 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 22:08:32.079952 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074336 5949 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 22:08:32.079952 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074342 5949 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 22:08:32.079952 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074347 5949 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 22:08:32.079952 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074351 5949 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 22:08:32.079952 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074355 5949 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 22:08:32.080771 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074360 5949 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 22:08:32.080771 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074364 5949 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 22:08:32.080771 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074368 5949 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 22:08:32.080771 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074373 5949 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 22:08:32.080771 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074377 5949 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 22:08:32.080771 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074381 5949 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 22:08:32.080771 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074385 5949 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 22:08:32.080771 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074389 5949 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 22:08:32.080771 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074393 5949 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 22:08:32.080771 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074398 5949 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 22:08:32.080771 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074402 5949 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 22:08:32.080771 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074407 5949 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 22:08:32.080771 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074411 5949 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 22:08:32.080771 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074417 5949 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 22:08:32.080771 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074421 5949 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 22:08:32.080771 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074425 5949 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 22:08:32.080771 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074430 5949 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 22:08:32.080771 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074434 5949 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 22:08:32.080771 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074439 5949 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 22:08:32.080771 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074443 5949 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 22:08:32.081426 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074448 5949 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 22:08:32.081426 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074452 5949 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 22:08:32.081426 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074457 5949 feature_gate.go:386] unrecognized feature gate: Example Oct 30 22:08:32.081426 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074461 5949 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 22:08:32.081426 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074465 5949 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 22:08:32.081426 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074470 5949 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 22:08:32.081426 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074474 5949 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 22:08:32.081426 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074478 5949 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 22:08:32.081426 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074482 5949 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 22:08:32.081426 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074486 5949 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 22:08:32.081426 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074490 5949 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 22:08:32.081426 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074494 5949 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 22:08:32.081426 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074498 5949 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 22:08:32.081426 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074502 5949 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 22:08:32.081426 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074507 5949 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 22:08:32.081426 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074511 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 22:08:32.081426 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074515 5949 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 22:08:32.081426 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074520 5949 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 22:08:32.081426 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074526 5949 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 22:08:32.081426 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074531 5949 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 22:08:32.081932 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074536 5949 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 22:08:32.081932 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074540 5949 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 22:08:32.081932 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074544 5949 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 22:08:32.081932 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074548 5949 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 22:08:32.081932 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074552 5949 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 22:08:32.081932 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074558 5949 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 22:08:32.081932 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074562 5949 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 22:08:32.081932 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074567 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 22:08:32.081932 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074571 5949 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 22:08:32.081932 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074576 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 22:08:32.081932 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074580 5949 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 22:08:32.081932 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074589 5949 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 22:08:32.081932 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074595 5949 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 22:08:32.081932 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074601 5949 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 22:08:32.081932 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074606 5949 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 22:08:32.081932 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074611 5949 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 22:08:32.081932 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074617 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 22:08:32.081932 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074621 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 22:08:32.081932 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074625 5949 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 22:08:32.082459 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.074629 5949 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 22:08:32.082459 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.074760 5949 flags.go:64] FLAG: --address="0.0.0.0" Oct 30 22:08:32.082459 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.074771 5949 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Oct 30 22:08:32.082459 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.074782 5949 flags.go:64] FLAG: --anonymous-auth="true" Oct 30 22:08:32.082459 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.074788 5949 flags.go:64] FLAG: --application-metrics-count-limit="100" Oct 30 22:08:32.082459 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.074796 5949 flags.go:64] FLAG: --authentication-token-webhook="false" Oct 30 22:08:32.082459 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.074801 5949 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Oct 30 22:08:32.082459 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.074808 5949 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Oct 30 22:08:32.082459 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.074815 5949 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Oct 30 22:08:32.082459 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.074820 5949 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Oct 30 22:08:32.082459 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.074825 5949 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Oct 30 22:08:32.082459 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.074833 5949 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Oct 30 22:08:32.082459 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.074838 5949 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Oct 30 22:08:32.082459 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.074844 5949 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Oct 30 22:08:32.082459 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.074848 5949 flags.go:64] FLAG: --cgroup-root="" Oct 30 22:08:32.082459 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.074853 5949 flags.go:64] FLAG: --cgroups-per-qos="true" Oct 30 22:08:32.082459 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.074859 5949 flags.go:64] FLAG: --client-ca-file="" Oct 30 22:08:32.082459 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.074864 5949 flags.go:64] FLAG: --cloud-provider="external" Oct 30 22:08:32.082459 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.074871 5949 flags.go:64] FLAG: --cluster-dns="[]" Oct 30 22:08:32.082459 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.074877 5949 flags.go:64] FLAG: --cluster-domain="" Oct 30 22:08:32.082459 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.074882 5949 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Oct 30 22:08:32.082459 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.074887 5949 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Oct 30 22:08:32.082459 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.074892 5949 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Oct 30 22:08:32.082459 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.074898 5949 flags.go:64] FLAG: --container-log-max-files="5" Oct 30 22:08:32.083050 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.074908 5949 flags.go:64] FLAG: --container-log-max-size="10Mi" Oct 30 22:08:32.083050 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.074913 5949 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Oct 30 22:08:32.083050 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.074919 5949 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Oct 30 22:08:32.083050 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.074924 5949 flags.go:64] FLAG: --containerd-namespace="k8s.io" Oct 30 22:08:32.083050 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.074929 5949 flags.go:64] FLAG: --contention-profiling="false" Oct 30 22:08:32.083050 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.074934 5949 flags.go:64] FLAG: --cpu-cfs-quota="true" Oct 30 22:08:32.083050 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.074939 5949 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Oct 30 22:08:32.083050 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.074944 5949 flags.go:64] FLAG: --cpu-manager-policy="none" Oct 30 22:08:32.083050 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.074949 5949 flags.go:64] FLAG: --cpu-manager-policy-options="" Oct 30 22:08:32.083050 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.074956 5949 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Oct 30 22:08:32.083050 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.074961 5949 flags.go:64] FLAG: --enable-controller-attach-detach="true" Oct 30 22:08:32.083050 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.074966 5949 flags.go:64] FLAG: --enable-debugging-handlers="true" Oct 30 22:08:32.083050 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.074971 5949 flags.go:64] FLAG: --enable-load-reader="false" Oct 30 22:08:32.083050 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.074976 5949 flags.go:64] FLAG: --enable-server="true" Oct 30 22:08:32.083050 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.074980 5949 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Oct 30 22:08:32.083050 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.074989 5949 flags.go:64] FLAG: --event-burst="100" Oct 30 22:08:32.083050 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.074994 5949 flags.go:64] FLAG: --event-qps="50" Oct 30 22:08:32.083050 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.074999 5949 flags.go:64] FLAG: --event-storage-age-limit="default=0" Oct 30 22:08:32.083050 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075004 5949 flags.go:64] FLAG: --event-storage-event-limit="default=0" Oct 30 22:08:32.083050 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075010 5949 flags.go:64] FLAG: --eviction-hard="" Oct 30 22:08:32.083050 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075017 5949 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Oct 30 22:08:32.083050 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075022 5949 flags.go:64] FLAG: --eviction-minimum-reclaim="" Oct 30 22:08:32.083050 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075027 5949 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Oct 30 22:08:32.083050 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075032 5949 flags.go:64] FLAG: --eviction-soft="" Oct 30 22:08:32.083050 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075037 5949 flags.go:64] FLAG: --eviction-soft-grace-period="" Oct 30 22:08:32.083728 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075042 5949 flags.go:64] FLAG: --exit-on-lock-contention="false" Oct 30 22:08:32.083728 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075048 5949 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Oct 30 22:08:32.083728 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075053 5949 flags.go:64] FLAG: --experimental-mounter-path="" Oct 30 22:08:32.083728 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075058 5949 flags.go:64] FLAG: --fail-cgroupv1="true" Oct 30 22:08:32.083728 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075062 5949 flags.go:64] FLAG: --fail-swap-on="true" Oct 30 22:08:32.083728 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075067 5949 flags.go:64] FLAG: --feature-gates="" Oct 30 22:08:32.083728 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075074 5949 flags.go:64] FLAG: --file-check-frequency="20s" Oct 30 22:08:32.083728 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075079 5949 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Oct 30 22:08:32.083728 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075084 5949 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Oct 30 22:08:32.083728 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075090 5949 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Oct 30 22:08:32.083728 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075095 5949 flags.go:64] FLAG: --healthz-port="10248" Oct 30 22:08:32.083728 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075100 5949 flags.go:64] FLAG: --help="false" Oct 30 22:08:32.083728 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075105 5949 flags.go:64] FLAG: --hostname-override="ip-10-0-142-24.ec2.internal" Oct 30 22:08:32.083728 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075111 5949 flags.go:64] FLAG: --housekeeping-interval="10s" Oct 30 22:08:32.083728 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075117 5949 flags.go:64] FLAG: --http-check-frequency="20s" Oct 30 22:08:32.083728 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075122 5949 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Oct 30 22:08:32.083728 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075127 5949 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Oct 30 22:08:32.083728 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075133 5949 flags.go:64] FLAG: --image-gc-high-threshold="85" Oct 30 22:08:32.083728 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075138 5949 flags.go:64] FLAG: --image-gc-low-threshold="80" Oct 30 22:08:32.083728 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075142 5949 flags.go:64] FLAG: --image-service-endpoint="" Oct 30 22:08:32.083728 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075147 5949 flags.go:64] FLAG: --kernel-memcg-notification="false" Oct 30 22:08:32.083728 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075152 5949 flags.go:64] FLAG: --kube-api-burst="100" Oct 30 22:08:32.083728 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075156 5949 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Oct 30 22:08:32.084301 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075162 5949 flags.go:64] FLAG: --kube-api-qps="50" Oct 30 22:08:32.084301 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075167 5949 flags.go:64] FLAG: --kube-reserved="" Oct 30 22:08:32.084301 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075172 5949 flags.go:64] FLAG: --kube-reserved-cgroup="" Oct 30 22:08:32.084301 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075178 5949 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Oct 30 22:08:32.084301 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075183 5949 flags.go:64] FLAG: --kubelet-cgroups="" Oct 30 22:08:32.084301 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075187 5949 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Oct 30 22:08:32.084301 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075192 5949 flags.go:64] FLAG: --lock-file="" Oct 30 22:08:32.084301 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075197 5949 flags.go:64] FLAG: --log-cadvisor-usage="false" Oct 30 22:08:32.084301 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075201 5949 flags.go:64] FLAG: --log-flush-frequency="5s" Oct 30 22:08:32.084301 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075206 5949 flags.go:64] FLAG: --log-json-info-buffer-size="0" Oct 30 22:08:32.084301 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075233 5949 flags.go:64] FLAG: --log-json-split-stream="false" Oct 30 22:08:32.084301 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075237 5949 flags.go:64] FLAG: --log-text-info-buffer-size="0" Oct 30 22:08:32.084301 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075242 5949 flags.go:64] FLAG: --log-text-split-stream="false" Oct 30 22:08:32.084301 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075246 5949 flags.go:64] FLAG: --logging-format="text" Oct 30 22:08:32.084301 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075250 5949 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Oct 30 22:08:32.084301 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075255 5949 flags.go:64] FLAG: --make-iptables-util-chains="true" Oct 30 22:08:32.084301 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075259 5949 flags.go:64] FLAG: --manifest-url="" Oct 30 22:08:32.084301 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075263 5949 flags.go:64] FLAG: --manifest-url-header="" Oct 30 22:08:32.084301 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075270 5949 flags.go:64] FLAG: --max-housekeeping-interval="15s" Oct 30 22:08:32.084301 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075276 5949 flags.go:64] FLAG: --max-open-files="1000000" Oct 30 22:08:32.084301 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075282 5949 flags.go:64] FLAG: --max-pods="110" Oct 30 22:08:32.084301 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075286 5949 flags.go:64] FLAG: --maximum-dead-containers="-1" Oct 30 22:08:32.084301 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075291 5949 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Oct 30 22:08:32.084301 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075296 5949 flags.go:64] FLAG: --memory-manager-policy="None" Oct 30 22:08:32.084301 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075301 5949 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Oct 30 22:08:32.084916 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075306 5949 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Oct 30 22:08:32.084916 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075311 5949 flags.go:64] FLAG: --node-ip="0.0.0.0" Oct 30 22:08:32.084916 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075316 5949 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Oct 30 22:08:32.084916 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075327 5949 flags.go:64] FLAG: --node-status-max-images="50" Oct 30 22:08:32.084916 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075332 5949 flags.go:64] FLAG: --node-status-update-frequency="10s" Oct 30 22:08:32.084916 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075337 5949 flags.go:64] FLAG: --oom-score-adj="-999" Oct 30 22:08:32.084916 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075341 5949 flags.go:64] FLAG: --pod-cidr="" Oct 30 22:08:32.084916 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075346 5949 flags.go:64] FLAG: --pod-manifest-path="" Oct 30 22:08:32.084916 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075351 5949 flags.go:64] FLAG: --pod-max-pids="-1" Oct 30 22:08:32.084916 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075356 5949 flags.go:64] FLAG: --pods-per-core="0" Oct 30 22:08:32.084916 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075362 5949 flags.go:64] FLAG: --port="10250" Oct 30 22:08:32.084916 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075367 5949 flags.go:64] FLAG: --protect-kernel-defaults="false" Oct 30 22:08:32.084916 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075372 5949 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f54b1358f274f8fa" Oct 30 22:08:32.084916 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075377 5949 flags.go:64] FLAG: --qos-reserved="" Oct 30 22:08:32.084916 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075382 5949 flags.go:64] FLAG: --read-only-port="10255" Oct 30 22:08:32.084916 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075387 5949 flags.go:64] FLAG: --register-node="true" Oct 30 22:08:32.084916 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075392 5949 flags.go:64] FLAG: --register-with-taints="" Oct 30 22:08:32.084916 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075401 5949 flags.go:64] FLAG: --registry-burst="10" Oct 30 22:08:32.084916 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075406 5949 flags.go:64] FLAG: --registry-qps="5" Oct 30 22:08:32.084916 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075411 5949 flags.go:64] FLAG: --reserved-cpus="" Oct 30 22:08:32.084916 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075415 5949 flags.go:64] FLAG: --reserved-memory="" Oct 30 22:08:32.084916 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075421 5949 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Oct 30 22:08:32.084916 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075426 5949 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Oct 30 22:08:32.084916 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075431 5949 flags.go:64] FLAG: --rotate-certificates="false" Oct 30 22:08:32.084916 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075435 5949 flags.go:64] FLAG: --rotate-server-certificates="false" Oct 30 22:08:32.084916 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075440 5949 flags.go:64] FLAG: --runonce="false" Oct 30 22:08:32.085572 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075445 5949 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Oct 30 22:08:32.085572 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075451 5949 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Oct 30 22:08:32.085572 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075456 5949 flags.go:64] FLAG: --seccomp-default="false" Oct 30 22:08:32.085572 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075460 5949 flags.go:64] FLAG: --serialize-image-pulls="true" Oct 30 22:08:32.085572 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075465 5949 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Oct 30 22:08:32.085572 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075471 5949 flags.go:64] FLAG: --storage-driver-db="cadvisor" Oct 30 22:08:32.085572 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075476 5949 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Oct 30 22:08:32.085572 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075481 5949 flags.go:64] FLAG: --storage-driver-password="root" Oct 30 22:08:32.085572 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075485 5949 flags.go:64] FLAG: --storage-driver-secure="false" Oct 30 22:08:32.085572 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075490 5949 flags.go:64] FLAG: --storage-driver-table="stats" Oct 30 22:08:32.085572 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075495 5949 flags.go:64] FLAG: --storage-driver-user="root" Oct 30 22:08:32.085572 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075499 5949 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Oct 30 22:08:32.085572 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075504 5949 flags.go:64] FLAG: --sync-frequency="1m0s" Oct 30 22:08:32.085572 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075509 5949 flags.go:64] FLAG: --system-cgroups="" Oct 30 22:08:32.085572 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075514 5949 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Oct 30 22:08:32.085572 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075523 5949 flags.go:64] FLAG: --system-reserved-cgroup="" Oct 30 22:08:32.085572 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075530 5949 flags.go:64] FLAG: --tls-cert-file="" Oct 30 22:08:32.085572 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075535 5949 flags.go:64] FLAG: --tls-cipher-suites="[]" Oct 30 22:08:32.085572 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075541 5949 flags.go:64] FLAG: --tls-min-version="" Oct 30 22:08:32.085572 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075546 5949 flags.go:64] FLAG: --tls-private-key-file="" Oct 30 22:08:32.085572 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075551 5949 flags.go:64] FLAG: --topology-manager-policy="none" Oct 30 22:08:32.085572 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075555 5949 flags.go:64] FLAG: --topology-manager-policy-options="" Oct 30 22:08:32.085572 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075560 5949 flags.go:64] FLAG: --topology-manager-scope="container" Oct 30 22:08:32.085572 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075567 5949 flags.go:64] FLAG: --v="2" Oct 30 22:08:32.085572 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075574 5949 flags.go:64] FLAG: --version="false" Oct 30 22:08:32.086183 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075581 5949 flags.go:64] FLAG: --vmodule="" Oct 30 22:08:32.086183 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075587 5949 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Oct 30 22:08:32.086183 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.075592 5949 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Oct 30 22:08:32.086183 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.075835 5949 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 22:08:32.086183 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.075842 5949 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 22:08:32.086183 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.075848 5949 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 22:08:32.086183 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.075852 5949 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 22:08:32.086183 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.075857 5949 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 22:08:32.086183 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.075862 5949 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 22:08:32.086183 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.075867 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 22:08:32.086183 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.075872 5949 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 22:08:32.086183 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.075876 5949 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 22:08:32.086183 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.075880 5949 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 22:08:32.086183 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.075884 5949 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 22:08:32.086183 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.075889 5949 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 22:08:32.086183 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.075894 5949 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 22:08:32.086183 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.075898 5949 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 22:08:32.086183 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.075903 5949 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 22:08:32.086183 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.075908 5949 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 22:08:32.086183 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.075912 5949 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 22:08:32.086717 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.075917 5949 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 22:08:32.086717 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.075921 5949 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 22:08:32.086717 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.075926 5949 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 22:08:32.086717 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.075932 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 22:08:32.086717 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.075937 5949 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 22:08:32.086717 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.075941 5949 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 22:08:32.086717 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.075946 5949 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 22:08:32.086717 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.075950 5949 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 22:08:32.086717 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.075954 5949 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 22:08:32.086717 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.075959 5949 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 22:08:32.086717 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.075965 5949 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 22:08:32.086717 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.075969 5949 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 22:08:32.086717 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.075974 5949 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 22:08:32.086717 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.075978 5949 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 22:08:32.086717 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.075985 5949 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 22:08:32.086717 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.075991 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 22:08:32.086717 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.075996 5949 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 22:08:32.086717 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076000 5949 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 22:08:32.086717 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076005 5949 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 22:08:32.087357 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076009 5949 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 22:08:32.087357 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076013 5949 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 22:08:32.087357 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076018 5949 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 22:08:32.087357 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076023 5949 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 22:08:32.087357 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076027 5949 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 22:08:32.087357 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076031 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 22:08:32.087357 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076036 5949 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 22:08:32.087357 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076041 5949 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 22:08:32.087357 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076045 5949 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 22:08:32.087357 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076049 5949 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 22:08:32.087357 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076053 5949 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 22:08:32.087357 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076058 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 22:08:32.087357 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076062 5949 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 22:08:32.087357 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076066 5949 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 22:08:32.087357 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076070 5949 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 22:08:32.087357 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076075 5949 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 22:08:32.087357 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076081 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 22:08:32.087357 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076085 5949 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 22:08:32.087357 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076090 5949 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 22:08:32.087357 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076094 5949 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 22:08:32.087964 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076099 5949 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 22:08:32.087964 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076105 5949 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 22:08:32.087964 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076111 5949 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 22:08:32.087964 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076119 5949 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 22:08:32.087964 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076125 5949 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 22:08:32.087964 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076130 5949 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 22:08:32.087964 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076134 5949 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 22:08:32.087964 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076139 5949 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 22:08:32.087964 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076143 5949 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 22:08:32.087964 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076148 5949 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 22:08:32.087964 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076152 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 22:08:32.087964 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076157 5949 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 22:08:32.087964 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076162 5949 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 22:08:32.087964 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076166 5949 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 22:08:32.087964 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076171 5949 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 22:08:32.087964 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076175 5949 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 22:08:32.087964 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076180 5949 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 22:08:32.087964 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076185 5949 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 22:08:32.087964 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076189 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 22:08:32.088461 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076194 5949 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 22:08:32.088461 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076198 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 22:08:32.088461 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076202 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 22:08:32.088461 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076207 5949 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 22:08:32.088461 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076211 5949 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 22:08:32.088461 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076231 5949 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 22:08:32.088461 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076235 5949 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 22:08:32.088461 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076240 5949 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 22:08:32.088461 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076245 5949 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 22:08:32.088461 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076251 5949 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 22:08:32.088461 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076255 5949 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 22:08:32.088461 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076259 5949 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 22:08:32.088461 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076264 5949 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 22:08:32.088461 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076268 5949 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 22:08:32.088461 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076273 5949 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 22:08:32.088461 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076277 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 22:08:32.088461 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076283 5949 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 22:08:32.088461 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076288 5949 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 22:08:32.088461 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076293 5949 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 22:08:32.088944 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076297 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 22:08:32.088944 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076301 5949 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 22:08:32.088944 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076306 5949 feature_gate.go:386] unrecognized feature gate: Example Oct 30 22:08:32.088944 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076310 5949 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 22:08:32.088944 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076315 5949 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 22:08:32.088944 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076319 5949 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 22:08:32.088944 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076324 5949 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 22:08:32.088944 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076328 5949 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 22:08:32.088944 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076333 5949 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 22:08:32.088944 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076337 5949 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 22:08:32.088944 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076342 5949 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 22:08:32.088944 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076347 5949 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 22:08:32.088944 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.076387 5949 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 22:08:32.088944 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076441 5949 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 22:08:32.088944 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076449 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 22:08:32.088944 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076453 5949 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 22:08:32.089409 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076458 5949 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 22:08:32.089409 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076462 5949 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 22:08:32.089409 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076466 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 22:08:32.089409 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076471 5949 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 22:08:32.089409 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076475 5949 feature_gate.go:386] unrecognized feature gate: Example Oct 30 22:08:32.089409 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076480 5949 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 22:08:32.089409 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076484 5949 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 22:08:32.089409 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076489 5949 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 22:08:32.089409 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076493 5949 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 22:08:32.089409 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076498 5949 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 22:08:32.089409 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076502 5949 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 22:08:32.089409 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076506 5949 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 22:08:32.089409 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076510 5949 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 22:08:32.089409 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076515 5949 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 22:08:32.089409 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076519 5949 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 22:08:32.089409 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076523 5949 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 22:08:32.089409 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076528 5949 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 22:08:32.089409 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076532 5949 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 22:08:32.089409 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076536 5949 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 22:08:32.089409 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076541 5949 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 22:08:32.090077 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076545 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 22:08:32.090077 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076550 5949 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 22:08:32.090077 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076554 5949 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 22:08:32.090077 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076558 5949 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 22:08:32.090077 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076563 5949 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 22:08:32.090077 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076567 5949 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 22:08:32.090077 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076572 5949 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 22:08:32.090077 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076577 5949 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 22:08:32.090077 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076582 5949 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 22:08:32.090077 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076587 5949 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 22:08:32.090077 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076592 5949 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 22:08:32.090077 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076596 5949 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 22:08:32.090077 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076601 5949 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 22:08:32.090077 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076606 5949 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 22:08:32.090077 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076617 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 22:08:32.090077 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076622 5949 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 22:08:32.090077 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076626 5949 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 22:08:32.090077 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076631 5949 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 22:08:32.090077 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076635 5949 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 22:08:32.090077 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076640 5949 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 22:08:32.090621 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076644 5949 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 22:08:32.090621 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076648 5949 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 22:08:32.090621 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076652 5949 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 22:08:32.090621 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076656 5949 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 22:08:32.090621 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076661 5949 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 22:08:32.090621 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076668 5949 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 22:08:32.090621 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076675 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 22:08:32.090621 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076681 5949 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 22:08:32.090621 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076686 5949 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 22:08:32.090621 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076690 5949 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 22:08:32.090621 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076694 5949 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 22:08:32.090621 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076699 5949 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 22:08:32.090621 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076703 5949 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 22:08:32.090621 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076707 5949 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 22:08:32.090621 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076711 5949 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 22:08:32.090621 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076715 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 22:08:32.090621 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076720 5949 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 22:08:32.090621 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076725 5949 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 22:08:32.090621 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076730 5949 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 22:08:32.091086 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076734 5949 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 22:08:32.091086 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076738 5949 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 22:08:32.091086 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076743 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 22:08:32.091086 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076748 5949 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 22:08:32.091086 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076752 5949 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 22:08:32.091086 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076756 5949 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 22:08:32.091086 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076761 5949 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 22:08:32.091086 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076765 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 22:08:32.091086 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076769 5949 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 22:08:32.091086 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076774 5949 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 22:08:32.091086 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076778 5949 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 22:08:32.091086 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076782 5949 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 22:08:32.091086 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076789 5949 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 22:08:32.091086 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076794 5949 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 22:08:32.091086 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076800 5949 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 22:08:32.091086 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076804 5949 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 22:08:32.091086 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076808 5949 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 22:08:32.091086 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076813 5949 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 22:08:32.091086 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076817 5949 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 22:08:32.091569 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076822 5949 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 22:08:32.091569 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076826 5949 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 22:08:32.091569 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076830 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 22:08:32.091569 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076837 5949 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 22:08:32.091569 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076842 5949 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 22:08:32.091569 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076846 5949 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 22:08:32.091569 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076850 5949 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 22:08:32.091569 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076855 5949 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 22:08:32.091569 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076859 5949 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 22:08:32.091569 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076864 5949 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 22:08:32.091569 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076869 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 22:08:32.091569 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076873 5949 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 22:08:32.091569 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076877 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 22:08:32.091569 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076881 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 22:08:32.091569 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076885 5949 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 22:08:32.091569 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076890 5949 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 22:08:32.091569 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076895 5949 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 22:08:32.091569 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076899 5949 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 22:08:32.091569 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076904 5949 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 22:08:32.092029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076910 5949 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 22:08:32.092029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076914 5949 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 22:08:32.092029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076918 5949 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 22:08:32.092029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076922 5949 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 22:08:32.092029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076927 5949 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 22:08:32.092029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.076931 5949 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 22:08:32.092029 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.081961 5949 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Oct 30 22:08:32.092029 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.081977 5949 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Oct 30 22:08:32.092029 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.081998 5949 watchdog_linux.go:95] "Systemd watchdog is not enabled" Oct 30 22:08:32.092029 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.082003 5949 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Oct 30 22:08:32.092029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082070 5949 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 22:08:32.092029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082075 5949 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 22:08:32.092029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082078 5949 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 22:08:32.092029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082082 5949 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 22:08:32.092029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082085 5949 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 22:08:32.092029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082088 5949 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 22:08:32.092029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082091 5949 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 22:08:32.092029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082094 5949 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 22:08:32.092029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082096 5949 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 22:08:32.092563 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082100 5949 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 22:08:32.092563 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082102 5949 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 22:08:32.092563 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082105 5949 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 22:08:32.092563 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082108 5949 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 22:08:32.092563 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082111 5949 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 22:08:32.092563 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082114 5949 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 22:08:32.092563 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082118 5949 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 22:08:32.092563 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082123 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 22:08:32.092563 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082126 5949 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 22:08:32.092563 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082129 5949 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 22:08:32.092563 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082133 5949 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 22:08:32.092563 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082135 5949 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 22:08:32.092563 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082138 5949 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 22:08:32.092563 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082141 5949 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 22:08:32.092563 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082145 5949 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 22:08:32.092563 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082149 5949 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 22:08:32.092563 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082152 5949 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 22:08:32.092563 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082155 5949 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 22:08:32.092563 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082158 5949 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 22:08:32.093044 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082167 5949 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 22:08:32.093044 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082170 5949 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 22:08:32.093044 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082173 5949 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 22:08:32.093044 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082175 5949 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 22:08:32.093044 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082178 5949 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 22:08:32.093044 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082181 5949 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 22:08:32.093044 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082184 5949 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 22:08:32.093044 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082187 5949 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 22:08:32.093044 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082190 5949 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 22:08:32.093044 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082192 5949 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 22:08:32.093044 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082195 5949 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 22:08:32.093044 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082198 5949 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 22:08:32.093044 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082201 5949 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 22:08:32.093044 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082203 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 22:08:32.093044 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082206 5949 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 22:08:32.093044 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082209 5949 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 22:08:32.093044 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082212 5949 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 22:08:32.093044 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082214 5949 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 22:08:32.093044 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082241 5949 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 22:08:32.093044 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082244 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 22:08:32.093755 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082248 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 22:08:32.093755 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082251 5949 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 22:08:32.093755 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082254 5949 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 22:08:32.093755 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082256 5949 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 22:08:32.093755 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082259 5949 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 22:08:32.093755 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082262 5949 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 22:08:32.093755 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082265 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 22:08:32.093755 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082268 5949 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 22:08:32.093755 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082270 5949 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 22:08:32.093755 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082273 5949 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 22:08:32.093755 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082276 5949 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 22:08:32.093755 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082278 5949 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 22:08:32.093755 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082281 5949 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 22:08:32.093755 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082284 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 22:08:32.093755 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082286 5949 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 22:08:32.093755 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082289 5949 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 22:08:32.093755 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082292 5949 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 22:08:32.093755 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082295 5949 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 22:08:32.093755 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082297 5949 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 22:08:32.094435 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082301 5949 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 22:08:32.094435 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082303 5949 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 22:08:32.094435 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082306 5949 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 22:08:32.094435 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082309 5949 feature_gate.go:386] unrecognized feature gate: Example Oct 30 22:08:32.094435 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082312 5949 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 22:08:32.094435 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082315 5949 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 22:08:32.094435 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082317 5949 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 22:08:32.094435 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082320 5949 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 22:08:32.094435 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082323 5949 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 22:08:32.094435 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082326 5949 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 22:08:32.094435 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082328 5949 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 22:08:32.094435 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082331 5949 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 22:08:32.094435 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082334 5949 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 22:08:32.094435 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082337 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 22:08:32.094435 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082340 5949 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 22:08:32.094435 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082342 5949 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 22:08:32.094435 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082345 5949 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 22:08:32.094435 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082348 5949 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 22:08:32.094435 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082350 5949 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 22:08:32.094435 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082353 5949 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 22:08:32.094999 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082355 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 22:08:32.094999 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082358 5949 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 22:08:32.094999 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082361 5949 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 22:08:32.094999 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082363 5949 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 22:08:32.094999 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082366 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 22:08:32.094999 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082368 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 22:08:32.094999 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082372 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 22:08:32.094999 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082374 5949 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 22:08:32.094999 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082378 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 22:08:32.094999 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082380 5949 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 22:08:32.094999 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082383 5949 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 22:08:32.094999 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082386 5949 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 22:08:32.094999 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082389 5949 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 22:08:32.094999 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082392 5949 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 22:08:32.094999 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082394 5949 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 22:08:32.094999 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082397 5949 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 22:08:32.094999 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082400 5949 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 22:08:32.094999 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082402 5949 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 22:08:32.094999 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082405 5949 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 22:08:32.095520 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.082433 5949 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 22:08:32.095520 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082607 5949 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 22:08:32.095520 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082612 5949 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 22:08:32.095520 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082615 5949 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 22:08:32.095520 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082618 5949 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 22:08:32.095520 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082621 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 22:08:32.095520 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082625 5949 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 22:08:32.095520 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082628 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 22:08:32.095520 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082630 5949 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 22:08:32.095520 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082633 5949 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 22:08:32.095520 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082636 5949 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 22:08:32.095520 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082639 5949 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 22:08:32.095520 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082641 5949 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 22:08:32.095520 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082644 5949 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 22:08:32.095520 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082647 5949 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 22:08:32.095520 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082650 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 22:08:32.095984 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082652 5949 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 22:08:32.095984 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082656 5949 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 22:08:32.095984 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082661 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 22:08:32.095984 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082664 5949 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 22:08:32.095984 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082667 5949 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 22:08:32.095984 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082669 5949 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 22:08:32.095984 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082672 5949 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 22:08:32.095984 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082675 5949 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 22:08:32.095984 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082678 5949 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 22:08:32.095984 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082680 5949 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 22:08:32.095984 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082683 5949 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 22:08:32.095984 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082686 5949 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 22:08:32.095984 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082688 5949 feature_gate.go:386] unrecognized feature gate: Example Oct 30 22:08:32.095984 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082691 5949 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 22:08:32.095984 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082694 5949 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 22:08:32.095984 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082696 5949 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 22:08:32.095984 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082699 5949 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 22:08:32.095984 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082702 5949 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 22:08:32.095984 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082704 5949 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 22:08:32.095984 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082707 5949 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 22:08:32.098074 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082709 5949 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 22:08:32.098074 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082712 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 22:08:32.098074 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082715 5949 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 22:08:32.098074 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082718 5949 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 22:08:32.098074 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082721 5949 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 22:08:32.098074 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082723 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 22:08:32.098074 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082726 5949 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 22:08:32.098074 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082729 5949 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 22:08:32.098074 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082731 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 22:08:32.098074 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082734 5949 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 22:08:32.098074 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082737 5949 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 22:08:32.098074 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082740 5949 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 22:08:32.098074 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082742 5949 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 22:08:32.098074 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082745 5949 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 22:08:32.098074 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082747 5949 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 22:08:32.098074 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082750 5949 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 22:08:32.098074 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082753 5949 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 22:08:32.098074 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082755 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 22:08:32.098074 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082760 5949 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 22:08:32.098564 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082763 5949 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 22:08:32.098564 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082766 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 22:08:32.098564 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082768 5949 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 22:08:32.098564 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082771 5949 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 22:08:32.098564 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082773 5949 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 22:08:32.098564 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082776 5949 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 22:08:32.098564 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082779 5949 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 22:08:32.098564 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082781 5949 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 22:08:32.098564 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082784 5949 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 22:08:32.098564 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082787 5949 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 22:08:32.098564 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082789 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 22:08:32.098564 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082793 5949 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 22:08:32.098564 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082796 5949 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 22:08:32.098564 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082799 5949 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 22:08:32.098564 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082802 5949 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 22:08:32.098564 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082805 5949 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 22:08:32.098564 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082807 5949 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 22:08:32.098564 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082810 5949 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 22:08:32.098564 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082813 5949 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 22:08:32.099029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082816 5949 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 22:08:32.099029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082819 5949 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 22:08:32.099029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082821 5949 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 22:08:32.099029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082824 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 22:08:32.099029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082826 5949 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 22:08:32.099029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082829 5949 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 22:08:32.099029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082832 5949 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 22:08:32.099029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082834 5949 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 22:08:32.099029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082836 5949 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 22:08:32.099029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082839 5949 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 22:08:32.099029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082842 5949 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 22:08:32.099029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082844 5949 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 22:08:32.099029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082848 5949 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 22:08:32.099029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082851 5949 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 22:08:32.099029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082854 5949 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 22:08:32.099029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082856 5949 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 22:08:32.099029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082859 5949 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 22:08:32.099029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082862 5949 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 22:08:32.099029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082865 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 22:08:32.099029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082867 5949 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 22:08:32.099629 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082870 5949 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 22:08:32.099629 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082872 5949 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 22:08:32.099629 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082875 5949 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 22:08:32.099629 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082877 5949 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 22:08:32.099629 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082880 5949 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 22:08:32.099629 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082882 5949 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 22:08:32.099629 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082885 5949 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 22:08:32.099629 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082888 5949 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 22:08:32.099629 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082890 5949 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 22:08:32.099629 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082893 5949 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 22:08:32.099629 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082895 5949 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 22:08:32.099629 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082899 5949 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 22:08:32.099629 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082903 5949 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 22:08:32.099629 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.082927 5949 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 22:08:32.099629 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082984 5949 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 22:08:32.099629 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082989 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 22:08:32.100029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082992 5949 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 22:08:32.100029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082995 5949 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 22:08:32.100029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.082998 5949 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 22:08:32.100029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083001 5949 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 22:08:32.100029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083004 5949 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 22:08:32.100029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083007 5949 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 22:08:32.100029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083009 5949 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 22:08:32.100029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083012 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 22:08:32.100029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083016 5949 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 22:08:32.100029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083021 5949 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 22:08:32.100029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083025 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 22:08:32.100029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083027 5949 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 22:08:32.100029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083030 5949 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 22:08:32.100029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083033 5949 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 22:08:32.100029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083036 5949 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 22:08:32.100029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083038 5949 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 22:08:32.100029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083041 5949 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 22:08:32.100029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083044 5949 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 22:08:32.100029 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083047 5949 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 22:08:32.100664 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083049 5949 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 22:08:32.100664 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083052 5949 feature_gate.go:386] unrecognized feature gate: Example Oct 30 22:08:32.100664 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083055 5949 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 22:08:32.100664 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083058 5949 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 22:08:32.100664 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083061 5949 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 22:08:32.100664 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083063 5949 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 22:08:32.100664 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083066 5949 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 22:08:32.100664 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083069 5949 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 22:08:32.100664 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083071 5949 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 22:08:32.100664 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083074 5949 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 22:08:32.100664 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083080 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 22:08:32.100664 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083082 5949 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 22:08:32.100664 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083085 5949 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 22:08:32.100664 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083088 5949 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 22:08:32.100664 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083090 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 22:08:32.100664 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083093 5949 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 22:08:32.100664 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083096 5949 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 22:08:32.100664 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083098 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 22:08:32.100664 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083101 5949 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 22:08:32.100664 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083104 5949 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 22:08:32.101166 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083107 5949 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 22:08:32.101166 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083111 5949 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 22:08:32.101166 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083114 5949 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 22:08:32.101166 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083117 5949 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 22:08:32.101166 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083119 5949 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 22:08:32.101166 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083122 5949 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 22:08:32.101166 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083124 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 22:08:32.101166 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083127 5949 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 22:08:32.101166 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083130 5949 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 22:08:32.101166 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083133 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 22:08:32.101166 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083135 5949 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 22:08:32.101166 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083138 5949 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 22:08:32.101166 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083140 5949 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 22:08:32.101166 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083143 5949 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 22:08:32.101166 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083146 5949 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 22:08:32.101166 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083148 5949 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 22:08:32.101166 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083151 5949 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 22:08:32.101166 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083154 5949 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 22:08:32.101166 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083156 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 22:08:32.101661 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083161 5949 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 22:08:32.101661 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083164 5949 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 22:08:32.101661 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083167 5949 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 22:08:32.101661 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083171 5949 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 22:08:32.101661 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083174 5949 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 22:08:32.101661 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083177 5949 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 22:08:32.101661 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083180 5949 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 22:08:32.101661 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083182 5949 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 22:08:32.101661 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083185 5949 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 22:08:32.101661 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083188 5949 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 22:08:32.101661 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083190 5949 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 22:08:32.101661 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083193 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 22:08:32.101661 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083196 5949 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 22:08:32.101661 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083198 5949 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 22:08:32.101661 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083202 5949 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 22:08:32.101661 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083205 5949 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 22:08:32.101661 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083207 5949 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 22:08:32.101661 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083210 5949 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 22:08:32.101661 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083213 5949 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 22:08:32.102159 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083228 5949 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 22:08:32.102159 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083232 5949 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 22:08:32.102159 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083234 5949 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 22:08:32.102159 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083237 5949 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 22:08:32.102159 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083240 5949 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 22:08:32.102159 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083243 5949 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 22:08:32.102159 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083245 5949 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 22:08:32.102159 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083248 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 22:08:32.102159 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083251 5949 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 22:08:32.102159 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083253 5949 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 22:08:32.102159 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083256 5949 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 22:08:32.102159 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083259 5949 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 22:08:32.102159 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083261 5949 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 22:08:32.102159 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083264 5949 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 22:08:32.102159 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083267 5949 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 22:08:32.102159 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083270 5949 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 22:08:32.102159 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083274 5949 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 22:08:32.102159 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083278 5949 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 22:08:32.102159 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083281 5949 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 22:08:32.102159 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083285 5949 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 22:08:32.102666 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083287 5949 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 22:08:32.102666 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083290 5949 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 22:08:32.102666 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083293 5949 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 22:08:32.102666 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083295 5949 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 22:08:32.102666 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083298 5949 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 22:08:32.102666 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083301 5949 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 22:08:32.102666 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:32.083304 5949 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 22:08:32.102666 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.083361 5949 server.go:957] "Client rotation is on, will bootstrap in background" Oct 30 22:08:32.102666 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.084333 5949 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary" Oct 30 22:08:32.102666 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.084384 5949 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem" Oct 30 22:08:32.102666 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.084561 5949 server.go:1014] "Starting client certificate rotation" Oct 30 22:08:32.102666 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.084698 5949 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 30 22:08:32.102666 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.084749 5949 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 22:02:21 +0000 UTC" deadline="2028-08-13 19:16:08.128564664 +0000 UTC" Oct 30 22:08:32.102666 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.084782 5949 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15669h7m36.043784019s" Oct 30 22:08:32.102666 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.085020 5949 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 22:08:32.102666 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.086116 5949 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 22:08:32.103058 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.087280 5949 log.go:25] "Validated CRI v1 runtime API" Oct 30 22:08:32.103058 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.089251 5949 log.go:25] "Validated CRI v1 image API" Oct 30 22:08:32.103058 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.089968 5949 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Oct 30 22:08:32.103058 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.092308 5949 fs.go:135] Filesystem UUIDs: map[34377297-fbd4-4393-8bed-a10a13622c62:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 7ddc0a6e-163a-46a0-824c-88dd7e6fd7ac:/dev/nvme0n1p4] Oct 30 22:08:32.103187 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.092324 5949 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0a6c5e5091712c18a370c9096a9488012885e7100be16362bfd46acebb21aee7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0a6c5e5091712c18a370c9096a9488012885e7100be16362bfd46acebb21aee7/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0ec135aec9bec53203b4fa8b33f029a49e2f2e4c576d202064835d2827d00338/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0ec135aec9bec53203b4fa8b33f029a49e2f2e4c576d202064835d2827d00338/userdata/shm major:0 minor:74 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1b3456bb2efcc685f0f73cec86542ed7117b7502a4cba412c802ad4c9d644ed6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1b3456bb2efcc685f0f73cec86542ed7117b7502a4cba412c802ad4c9d644ed6/userdata/shm major:0 minor:514 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1f0618e251474d6ffb7acd4a8504e72915215cd2c4d8e67c1e553cced27ae6d2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1f0618e251474d6ffb7acd4a8504e72915215cd2c4d8e67c1e553cced27ae6d2/userdata/shm major:0 minor:349 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2320e8be4a7af79dbed9ac74a3043708a1694e6a3e6ecd5a21a60e433f61e3b5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2320e8be4a7af79dbed9ac74a3043708a1694e6a3e6ecd5a21a60e433f61e3b5/userdata/shm major:0 minor:310 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2e1024a48494975a37cd70c11675061811824847d6e8d357b5b6836a0f397c37/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2e1024a48494975a37cd70c11675061811824847d6e8d357b5b6836a0f397c37/userdata/shm major:0 minor:460 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/340578bb6c446168bf7e528fda076049b3664bf8c0ce11cb85be81554dd5d793/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/340578bb6c446168bf7e528fda076049b3664bf8c0ce11cb85be81554dd5d793/userdata/shm major:0 minor:239 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/36051ba353f4d1493e47afd3bee1fa39336698eac55c1ddd7a97c40dbd52bdfa/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/36051ba353f4d1493e47afd3bee1fa39336698eac55c1ddd7a97c40dbd52bdfa/userdata/shm major:0 minor:61 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4caf4e2411d03431dff003062637fa6d121bfffb6fc2c67e46afdbad23627e6a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4caf4e2411d03431dff003062637fa6d121bfffb6fc2c67e46afdbad23627e6a/userdata/shm major:0 minor:237 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4e612a431c19e72aa48209c41de2f70414134eda7dbfd4fe026be056f9641119/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4e612a431c19e72aa48209c41de2f70414134eda7dbfd4fe026be056f9641119/userdata/shm major:0 minor:274 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4fbf0fd61fc6b065e8ca1ffc84f42505b9d8905160540d9402b8fc358977448c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4fbf0fd61fc6b065e8ca1ffc84f42505b9d8905160540d9402b8fc358977448c/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/50767516a7f3953ef0ef84dec6c641854ecd806e5f5a3b2c0f75d7e46408cc12/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/50767516a7f3953ef0ef84dec6c641854ecd806e5f5a3b2c0f75d7e46408cc12/userdata/shm major:0 minor:207 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/61a90835359d781127b7c1b0c8148cf6aef8e3c7ad74148743763ac42eb9b7ad/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/61a90835359d781127b7c1b0c8148cf6aef8e3c7ad74148743763ac42eb9b7ad/userdata/shm major:0 minor:60 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/62773096baa81d57351c85102abff61a5273833dd16c64c61a2dbbc4c56411dd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/62773096baa81d57351c85102abff61a5273833dd16c64c61a2dbbc4c56411dd/userdata/shm major:0 minor:241 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/62c19734cc22ee07d6a2c049f1ee538a83cb1a6158f7b1dde8efdf1c7f7767f6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/62c19734cc22ee07d6a2c049f1ee538a83cb1a6158f7b1dde8efdf1c7f7767f6/userdata/shm major:0 minor:209 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7062c9becc10226a3ca19a89d297a7269e2c51a6b6b7dca0f430f14dcd410149/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7062c9becc10226a3ca19a89d297a7269e2c51a6b6b7dca0f430f14dcd410149/userdata/shm major:0 minor:353 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/81b300842b95d3636a9caf7f46ce2fbcf3b083baa3f13105be77089783c54aeb/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/81b300842b95d3636a9caf7f46ce2fbcf3b083baa3f13105be77089783c54aeb/userdata/shm major:0 minor:78 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/89137e0ed84568ba01d3ac9b0e14fd3577d3f7a7d8a68c4b3d1a82cc43fc6601/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/89137e0ed84568ba01d3ac9b0e14fd3577d3f7a7d8a68c4b3d1a82cc43fc6601/userdata/shm major:0 minor:70 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/976467f0e8d8d1bfa25ef9bd6e507f4786388b419da2bcea0765ec9a620aba24/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/976467f0e8d8d1bfa25ef9bd6e507f4786388b419da2bcea0765ec9a620aba24/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a432ddbd05d471d3613077a1a33b72f9dc711e4b0edb4221802c3fff94c648b8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a432ddbd05d471d3613077a1a33b72f9dc711e4b0edb4221802c3fff94c648b8/userdata/shm major:0 minor:335 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a7b56859943b84b7b61cb86d28e1f5cb2bced2688531f8d082c00371d84af8f9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a7b56859943b84b7b61cb86d28e1f5cb2bced2688531f8d082c00371d84af8f9/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/cd91dfa17aa2f5dbb8bf2c7c3f923a026fe1805f799b2dd62fe87019e250f503/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/cd91dfa17aa2f5dbb8bf2c7c3f923a026fe1805f799b2dd62fe87019e250f503/userdata/shm major:0 minor:398 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d71deada5e7ff6a92740256adf5084afe01473c75b9427829a90d8b0c608cd3a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d71deada5e7ff6a92740256adf5084afe01473c75b9427829a90d8b0c608cd3a/userdata/shm major:0 minor:518 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e77edf5e5faa5574836fffe4f9ac84164a97e19598785877baa1d628cf1b0a58/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e77edf5e5faa5574836fffe4f9ac84164a97e19598785877baa1d628cf1b0a58/userdata/shm major:0 minor:205 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/eb20dbc00c4333ecb219b11f019b76d9cd02e2de6867ca00a83081ff0adc9951/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/eb20dbc00c4333ecb219b11f019b76d9cd02e2de6867ca00a83081ff0adc9951/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f70f76b15ecaf9d5fff82369559697fc0109804c1f08a91ec6e029cb3acea159/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f70f76b15ecaf9d5fff82369559697fc0109804c1f08a91ec6e029cb3acea159/userdata/shm major:0 minor:515 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/074eeae7-932c-4642-8fea-671898135ab1/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/074eeae7-932c-4642-8fea-671898135ab1/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/074eeae7-932c-4642-8fea-671898135ab1/volumes/kubernetes.io~projected/kube-api-access-8kxhz:{mountpoint:/var/lib/kubelet/pods/074eeae7-932c-4642-8fea-671898135ab1/volumes/kubernetes.io~projected/kube-api-access-8kxhz major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/074eeae7-932c-4642-8fea-671898135ab1/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/074eeae7-932c-4642-8fea-671898135ab1/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:49 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/11464051-0aa4-4324-b2c7-7ea2096bbed6/volumes/kubernetes.io~projected/kube-api-access-nrwhs:{mountpoint:/var/lib/kubelet/pods/11464051-0aa4-4324-b2c7-7ea2096bbed6/volumes/kubernetes.io~projected/kube-api-access-nrwhs major:0 minor:51 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1b444016-092c-472b-979a-4a7b16271e27/volumes/kubernetes.io~projected/kube-api-access-j9862:{mountpoint:/var/lib/kubelet/pods/1b444016-092c-472b-979a-4a7b16271e27/volumes/kubernetes.io~projected/kube-api-access-j9862 major:0 minor:336 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1b444016-092c-472b-979a-4a7b16271e27/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/1b444016-092c-472b-979a-4a7b16271e27/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:334 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1b444016-092c-472b-979a-4a7b16271e27/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/1b444016-092c-472b-979a-4a7b16271e27/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy major:0 minor:329 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1b444016-092c-472b-979a-4a7b16271e27/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics:{mountpoint:/var/lib/kubelet/pods/1b444016-092c-472b-979a-4a7b16271e27/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics major:0 minor:332 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1b444016-092c-472b-979a-4a7b16271e27/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules:{mountpoint:/var/lib/kubelet/pods/1b444016-092c-472b-979a-4a7b16271e27/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules major:0 minor:331 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1b444016-092c-472b-979a-4a7b16271e27/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/1b444016-092c-472b-979a-4a7b16271e27/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web major:0 minor:333 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1b444016-092c-472b-979a-4a7b16271e27/volumes/kubernetes.io~secret/secret-thanos-querier-tls:{mountpoint:/var/lib/kubelet/pods/1b444016-092c-472b-979a-4a7b16271e27/volumes/kubernetes.io~secret/secret-thanos-querier-tls major:0 minor:330 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3155dfa9-0dc2-4256-9e16-490b3721702d/volumes/kubernetes.io~projected/kube-api-access-2gvkg:{mountpoint:/var/lib/kubelet/pods/3155dfa9-0dc2-4256-9e16-490b3721702d/volumes/kubernetes.io~projected/kube-api-access-2gvkg major:0 minor:513 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volume-subpaths/web-config/config-reloader/2:{mountpoint:/var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volume-subpaths/web-config/config-reloader/2 major:0 minor:456 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volume-subpaths/web-config/init-config-reloader/2:{mountpoint:/var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volume-subpaths/web-config/init-config-reloader/2 major:0 minor:456 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volume-subpaths/web-config/prometheus/15:{mountpoint:/var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volume-subpaths/web-config/prometheus/15 major:0 minor:456 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volumes/kubernetes.io~empty-dir/config-out major:0 minor:450 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volumes/kubernetes.io~projected/kube-api-access-qsvp8:{mountpoint:/var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volumes/kubernetes.io~projected/kube-api-access-qsvp8 major:0 minor:459 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volumes/kubernetes.io~projected/tls-assets major:0 minor:455 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volumes/kubernetes.io~secret/config:{mountpoint:/var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volumes/kubernetes.io~secret/config major:0 minor:441 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:457 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volumes/kubernetes.io~secret/secret-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volumes/kubernetes.io~secret/secret-kube-rbac-proxy major:0 minor:458 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volumes/kubernetes.io~secret/secret-metrics-client-certs:{mountpoint:/var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volumes/kubernetes.io~secret/secret-metrics-client-certs major:0 minor:453 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web major:0 minor:454 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls:{mountpoint:/var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls major:0 minor:440 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls:{mountpoint:/var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls major:0 minor:451 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file:{mountpoint:/var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file major:0 minor:452 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volumes/kubernetes.io~secret/web-config major:0 minor:456 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/46b06955-7c3e-4a10-b5cd-5c2f4c34859f/volume-subpaths/cluster-tls-config/alertmanager/10:{mountpoint:/var/lib/kubelet/pods/46b06955-7c3e-4a10-b5cd-5c2f4c34859f/volume-subpaths/cluster-tls-config/alertmanager/10 major:0 minor:282 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/46b06955-7c3e-4a10-b5cd-5c2f4c34859f/volume-subpaths/web-config/alertmanager/9:{mountpoint:/var/lib/kubelet/pods/46b06955-7c3e-4a10-b5cd-5c2f4c34859f/volume-subpaths/web-config/alertmanager/9 major:0 minor:278 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/46b06955-7c3e-4a10-b5cd-5c2f4c34859f/volume-subpaths/web-config/config-reloader/6:{mountpoint:/var/lib/kubelet/pods/46b06955-7c3e-4a10-b5cd-5c2f4c34859f/volume-subpaths/web-config/config-reloader/6 major:0 minor:278 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/46b06955-7c3e-4a10-b5cd-5c2f4c34859f/volume-subpaths/web-config/init-config-reloader/6:{mountpoint:/var/lib/kubelet/pods/46b06955-7c3e-4a10-b5cd-5c2f4c34859f/volume-subpaths/web-config/init-config-reloader/6 major:0 minor:278 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/46b06955-7c3e-4a10-b5cd-5c2f4c34859f/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/46b06955-7c3e-4a10-b5cd-5c2f4c34859f/volumes/kubernetes.io~empty-dir/config-out major:0 minor:286 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/46b06955-7c3e-4a10-b5cd-5c2f4c34859f/volumes/kubernetes.io~projected/kube-api-access-bpcht:{mountpoint:/var/lib/kubelet/pods/46b06955-7c3e-4a10-b5cd-5c2f4c34859f/volumes/kubernetes.io~projected/kube-api-access-bpcht major:0 minor:288 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/46b06955-7c3e-4a10-b5cd-5c2f4c34859f/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/46b06955-7c3e-4a10-b5cd-5c2f4c34859f/volumes/kubernetes.io~projected/tls-assets major:0 minor:283 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/46b06955-7c3e-4a10-b5cd-5c2f4c34859f/volumes/kubernetes.io~secret/cluster-tls-config:{mountpoint:/var/lib/kubelet/pods/46b06955-7c3e-4a10-b5cd-5c2f4c34859f/volumes/kubernetes.io~secret/cluster-tls-config major:0 minor:282 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/46b06955-7c3e-4a10-b5cd-5c2f4c34859f/volumes/kubernetes.io~secret/config-volume:{mountpoint:/var/lib/kubelet/pods/46b06955-7c3e-4a10-b5cd-5c2f4c34859f/volumes/kubernetes.io~secret/config-volume major:0 minor:279 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/46b06955-7c3e-4a10-b5cd-5c2f4c34859f/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/46b06955-7c3e-4a10-b5cd-5c2f4c34859f/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy major:0 minor:287 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/46b06955-7c3e-4a10-b5cd-5c2f4c34859f/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric:{mountpoint:/var/lib/kubelet/pods/46b06955-7c3e-4a10-b5cd-5c2f4c34859f/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric major:0 minor:285 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/46b06955-7c3e-4a10-b5cd-5c2f4c34859f/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/46b06955-7c3e-4a10-b5cd-5c2f4c34859f/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web major:0 minor:281 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/46b06955-7c3e-4a10-b5cd-5c2f4c34859f/volumes/kubernetes.io~secret/secret-alertmanager-main-tls:{mountpoint:/var/lib/kubelet/pods/46b06955-7c3e-4a10-b5cd-5c2f4c34859f/volumes/kubernetes.io~secret/secret-alertmanager-main-tls major:0 minor:284 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/46b06955-7c3e-4a10-b5cd-5c2f4c34859f/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/46b06955-7c3e-4a10-b5cd-5c2f4c34859f/volumes/kubernetes.io~secret/web-config major:0 minor:278 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5bb239d5-4f0e-4dd6-b58b-5f3b9e014895/volumes/kubernetes.io~projected/kube-api-access-knh4d:{mountpoint:/var/lib/kubelet/pods/5bb239d5-4f0e-4dd6-b58b-5f3b9e014895/volumes/kubernetes.io~projected/kube-api-access-knh4d major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5dfaf077-3637-4cc9-8681-466c4e2cc5f4/volumes/kubernetes.io~projected/kube-api-access-8r24v:{mountpoint:/var/lib/kubelet/pods/5dfaf077-3637-4cc9-8681-466c4e2cc5f4/volumes/kubernetes.io~projected/kube-api-access-8r24v major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5dfaf077-3637-4cc9-8681-466c4e2cc5f4/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/5dfaf077-3637-4cc9-8681-466c4e2cc5f4/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/65c2ef45-7211-4828-86bc-c9d7af39ec66/volumes/kubernetes.io~projected/kube-api-access-dpz88:{mountpoint:/var/lib/kubelet/pods/65c2ef45-7211-4828-86bc-c9d7af39ec66/volumes/kubernetes.io~projected/kube-api-access-dpz88 major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/65c2ef45-7211-4828-86bc-c9d7af39ec66/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/65c2ef45-7211-4828-86bc-c9d7af39ec66/volumes/kubernetes.io~secret/metrics-certs major:0 minor:508 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/68935e75-a4f0-4a54-9094-4f12b4a2dc39/volumes/kubernetes.io~projected/kube-api-access-z6fv5:{mountpoint:/var/lib/kubelet/pods/68935e75-a4f0-4a54-9094-4f12b4a2dc39/volumes/kubernetes.io~projected/kube-api-access-z6fv5 major:0 minor:236 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/68935e75-a4f0-4a54-9094-4f12b4a2dc39/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/68935e75-a4f0-4a54-9094-4f12b4a2dc39/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:235 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/68935e75-a4f0-4a54-9094-4f12b4a2dc39/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/68935e75-a4f0-4a54-9094-4f12b4a2dc39/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:234 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6b042dce-1fff-41a2-83dd-ae18f3098207/volumes/kubernetes.io~projected/kube-api-access-qxdbc:{mountpoint:/var/lib/kubelet/pods/6b042dce-1fff-41a2-83dd-ae18f3098207/volumes/kubernetes.io~projected/kube-api-access-qxdbc major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/94283f0e-a719-4152-aea8-e219f7bb7eef/volumes/kubernetes.io~projected/kube-api-access-k2kkt:{mountpoint:/var/lib/kubelet/pods/94283f0e-a719-4152-aea8-e219f7bb7eef/volumes/kubernetes.io~projected/kube-api-access-k2kkt major:0 minor:307 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/94283f0e-a719-4152-aea8-e219f7bb7eef/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/94283f0e-a719-4152-aea8-e219f7bb7eef/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:306 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/94283f0e-a719-4152-aea8-e219f7bb7eef/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/94283f0e-a719-4152-aea8-e219f7bb7eef/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:305 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9ac70a73-f7f0-4b3e-bf76-09869caefea0/volumes/kubernetes.io~projected/kube-api-access-bfsr2:{mountpoint:/var/lib/kubelet/pods/9ac70a73-f7f0-4b3e-bf76-09869caefea0/volumes/kubernetes.io~projected/kube-api-access-bfsr2 major:0 minor:348 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9ac70a73-f7f0-4b3e-bf76-09869caefea0/volumes/kubernetes.io~secret/client-ca-bundle:{mountpoint:/var/lib/kubelet/pods/9ac70a73-f7f0-4b3e-bf76-09869caefea0/volumes/kubernetes.io~secret/client-ca-bundle major:0 minor:347 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9ac70a73-f7f0-4b3e-bf76-09869caefea0/volumes/kubernetes.io~secret/secret-metrics-server-client-certs:{mountpoint:/var/lib/kubelet/pods/9ac70a73-f7f0-4b3e-bf76-09869caefea0/volumes/kubernetes.io~secret/secret-metrics-server-client-certs major:0 minor:346 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9ac70a73-f7f0-4b3e-bf76-09869caefea0/volumes/kubernetes.io~secret/secret-metrics-server-tls:{mountpoint:/var/lib/kubelet/pods/9ac70a73-f7f0-4b3e-bf76-09869caefea0/volumes/kubernetes.io~secret/secret-metrics-server-tls major:0 minor:345 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9c0d0893-e78b-48f3-bbe8-9f7d0381f42e/volumes/kubernetes.io~projected/kube-api-access-hxr72:{mountpoint:/var/lib/kubelet/pods/9c0d0893-e78b-48f3-bbe8-9f7d0381f42e/volumes/kubernetes.io~projected/kube-api-access-hxr72 major:0 minor:352 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9c0d0893-e78b-48f3-bbe8-9f7d0381f42e/volumes/kubernetes.io~secret/monitoring-plugin-cert:{mountpoint:/var/lib/kubelet/pods/9c0d0893-e78b-48f3-bbe8-9f7d0381f42e/volumes/kubernetes.io~secret/monitoring-plugin-cert major:0 minor:351 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8/volumes/kubernetes.io~projected/kube-api-access-hzvcp:{mountpoint:/var/lib/kubelet/pods/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8/volumes/kubernetes.io~projected/kube-api-access-hzvcp major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c373c767-d743-4819-be7c-4c9c830eb9db/volumes/kubernetes.io~projected/kube-api-access-cj545:{mountpoint:/var/lib/kubelet/pods/c373c767-d743-4819-be7c-4c9c830eb9db/volumes/kubernetes.io~projected/kube-api-access-cj545 major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c373c767-d743-4819-be7c-4c9c830eb9db/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/c373c767-d743-4819-be7c-4c9c830eb9db/volumes/kubernetes.io~secret/metrics-tls major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c9381d26-e2af-4730-8b4e-06945c59585c/volumes/kubernetes.io~projected/kube-api-access-xf5c4:{mountpoint:/var/lib/kubelet/pods/c9381d26-e2af-4730-8b4e-06945c59585c/volumes/kubernetes.io~projected/kube-api-access-xf5c4 major:0 minor:52 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cd46f3e8-c752-4eed-a65d-07e060bf32b4/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/cd46f3e8-c752-4eed-a65d-07e060bf32b4/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:512 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/df6b6cae-173c-461c-9195-4dce90bb6656/volumes/kubernetes.io~projected/kube-api-access-45l5v:{mountpoint:/var/lib/kubelet/pods/df6b6cae-173c-461c-9195-4dce90bb6656/volumes/kubernetes.io~projected/kube-api-access-45l5v major:0 minor:232 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/df6b6cae-173c-461c-9195-4dce90bb6656/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/df6b6cae-173c-461c-9195-4dce90bb6656/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config major:0 minor:230 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/df6b6cae-173c-461c-9195-4dce90bb6656/volumes/kubernetes.io~secret/kube-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/df6b6cae-173c-461c-9195-4dce90bb6656/volumes/kubernetes.io~secret/kube-state-metrics-tls major:0 minor:231 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e129700d-0583-4316-bcee-431c57b0effa/volumes/kubernetes.io~projected/kube-api-access-5zxs5:{mountpoint:/var/lib/kubelet/pods/e129700d-0583-4316-bcee-431c57b0effa/volumes/kubernetes.io~projected/kube-api-access-5zxs5 major:0 minor:233 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e129700d-0583-4316-bcee-431c57b0effa/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/e129700d-0583-4316-bcee-431c57b0effa/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e129700d-0583-4316-bcee-431c57b0effa/volumes/kubernetes.io~secret/openshift-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/e129700d-0583-4316-bcee-431c57b0effa/volumes/kubernetes.io~secret/openshift-state-metrics-tls major:0 minor:229 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e5779bc8-21ad-465e-8749-8f7e9afdc03f/volumes/kubernetes.io~projected/kube-api-access-h7f59:{mountpoint:/var/lib/kubelet/pods/e5779bc8-21ad-465e-8749-8f7e9afdc03f/volumes/kubernetes.io~projected/kube-api-access-h7f59 major:0 minor:53 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:50 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44/volumes/kubernetes.io~empty-dir/tmp major:0 minor:48 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44/volumes/kubernetes.io~projected/kube-api-access-2prlk:{mountpoint:/var/lib/kubelet/pods/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44/volumes/kubernetes.io~projected/kube-api-access-2prlk major:0 minor:54 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f64980b7-13c0-40c3-aa3f-277916f24518/volumes/kubernetes.io~projected/kube-api-access-tjk5b:{mountpoint:/var/lib/kubelet/pods/f64980b7-13c0-40c3-aa3f-277916f24518/volumes/kubernetes.io~projected/kube-api-access-tjk5b major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f64980b7-13c0-40c3-aa3f-277916f24518/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/f64980b7-13c0-40c3-aa3f-277916f24518/volumes/kubernetes.io~secret/cert major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f6cc017b-3f50-423a-93da-e4515336a2bc/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/f6cc017b-3f50-423a-93da-e4515336a2bc/volumes/kubernetes.io~secret/agent-certs major:0 minor:47 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fb3d3e48-51f2-4503-8026-79787ee98754/volumes/kubernetes.io~projected/kube-api-access-jtbtv:{mountpoint:/var/lib/kubelet/pods/fb3d3e48-51f2-4503-8026-79787ee98754/volumes/kubernetes.io~projected/kube-api-access-jtbtv major:0 minor:392 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fb3d3e48-51f2-4503-8026-79787ee98754/volumes/kubernetes.io~secret/federate-client-tls:{mountpoint:/var/lib/kubelet/pods/fb3d3e48-51f2-4503-8026-79787ee98754/volumes/kubernetes.io~secret/federate-client-tls major:0 minor:390 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fb3d3e48-51f2-4503-8026-79787ee98754/volumes/kubernetes.io~secret/secret-telemeter-client:{mountpoint:/var/lib/kubelet/pods/fb3d3e48-51f2-4503-8026-79787ee98754/volumes/kubernetes.io~secret/secret-telemeter-client major:0 minor:383 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fb3d3e48-51f2-4503-8026-79787ee98754/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/fb3d3e48-51f2-4503-8026-79787ee98754/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config major:0 minor:391 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fb3d3e48-51f2-4503-8026-79787ee98754/volumes/kubernetes.io~secret/telemeter-client-tls:{mountpoint:/var/lib/kubelet/pods/fb3d3e48-51f2-4503-8026-79787ee98754/volumes/kubernetes.io~secret/telemeter-client-tls major:0 minor:380 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/a5edb13b8f94a7902e406d1bb75d74770354a6132a92773248e6bbffa32a4723/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/c7451d654ba36267142919397c381ff27132de4a18d198d6d790a1ec044ef3b9/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/3ed345862b25ddbd83b2a5678e2802e782b1e0ce0086bc4a814d234752c4dd02/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/b12ca62f29df2f60136c463f3790960cb7a8f3a3cf742cadf82ef71f0eb1b78d/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/1bb814c8f356e61647ba4638926a15f66fda18f25be36570191d887c32807799/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/3cfda9d6eab12436dbd76e5525dff2c9363b98671c49f819a02ffc41aafeae02/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/d00b8bc812de84607f21a3b607843cd49cafcc0ac5331ef7011e04d83edeab9a/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/a3f080a828c745477a6af6076aa294c0db81bcfcdbfbb7f40476cb4cf6a209db/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/147af378d3502ff56481a608c988304209a78c1d92c2f7ea69b747728af78393/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/e3487ad844f3bb46bb669a3290f3439ee64660184e79d1acfbaae05ceb45eaff/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-138:{mountpoint:/var/lib/containers/storage/overlay/d0f29c1586557caafc4f59271f0bd5d5d353e563328199fc24aab937527ef632/merged major:0 minor:138 fsType:overlay blockSize:0} overlay_0-147:{mountpoint:/var/lib/containers/storage/overlay/4d441054109fe32e33f0f8554043851d8823ae5157859dd6ad9929afdc545417/merged major:0 minor:147 fsType:overlay blockSize:0} overlay_0-148:{mountpoint:/var/lib/containers/storage/overlay/4289e1f0648d44f2a77dad9f6cae408893b5ac9de0c30e52ea0bba135ebd120c/merged major:0 minor:148 fsType:overlay blockSize:0} overlay_0-151:{mountpoint:/var/lib/containers/storage/overlay/60b55576c845cd94afda04cb80b79f3658923a9e3d92b717fbd4943c819be724/merged major:0 minor:151 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/1202592978f31bbefc8abdf69ef627a778559e900f2b10cf058880d8f3a402b3/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/0ad5563ef6075093076910da30198d695b4bdba58ec31260e7787931cc0fbb53/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/b7582f6b461713a73e54df67476c7c8618dffeaf4e9fac0633a3d547255b1bfb/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/ffe6ce405b5dd10076a46808eda4dd6bd2b964202ffe6147371391ba51ec59b3/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/3f5770117c4e1502f00fc09a0a1ed72997bd2cc6a21458edf5f8cedb9fa4911d/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/c7862e649d9811c1fb9872909c5bc79f579668cb77c37990c94d5a8fd848d99f/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/7cc40851ebe9e861db7d0b00d5247413f1cfcc76f005f6a545d59a3fc3f9760f/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/2a0855dc29824db9484ba20665358597825da4c50364acc76b5270924edaee9a/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-213:{mountpoint:/var/lib/containers/storage/overlay/0edbb0a5a17563f995d909577f9141d1c0693555683e6675551b6de0cdad516e/merged major:0 minor:213 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/c376200cdd9fa3135be04dd7667ae8b375ef67ab4e65419ff9f93a8bb284249c/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-217:{mountpoint:/var/lib/containers/storage/overlay/f5ba595602d7de5e50886b177bfd8f0b387fcc4195b48570fc89c75dfb36dbb0/merged major:0 minor:217 fsType:overlay blockSize:0} overlay_0-219:{mountpoint:/var/lib/containers/storage/overlay/d7f5b50f509eb076b54feaebe48d6a22da279473bf454c68d17d3dac07e73192/merged major:0 minor:219 fsType:overlay blockSize:0} overlay_0-243:{mountpoint:/var/lib/containers/storage/overlay/1f454e61effe932a9e333721491aedaaa709b8d9c71274c45fd3220588705665/merged major:0 minor:243 fsType:overlay blockSize:0} overlay_0-245:{mountpoint:/var/lib/containers/storage/overlay/9ba9683e4227201e6f430fd628b2d78b412340893d9ac28d4dafc31b6f34bb30/merged major:0 minor:245 fsType:overlay blockSize:0} overlay_0-247:{mountpoint:/var/lib/containers/storage/overlay/87c5f2a4866ecde835577a1769087c974f4ad4422d4033869fe725562bb2afa2/merged major:0 minor:247 fsType:overlay blockSize:0} overlay_0-249:{mountpoint:/var/lib/containers/storage/overlay/961ba176df5ac69ae79e215fc31606fbe6cddca52989a99e4cd0423720c3cb21/merged major:0 minor:249 fsType:overlay blockSize:0} overlay_0-255:{mountpoint:/var/lib/containers/storage/overlay/f8f03a707ff2d2a09d36d6fdd9a9021e4f1dc0331caecfffa6727be7817c035e/merged major:0 minor:255 fsType:overlay blockSize:0} overlay_0-257:{mountpoint:/var/lib/containers/storage/overlay/f61ac47d29aab29b9be42ac3b428f625c13cdca2fb408a69bed54f6ab38209e5/merged major:0 minor:257 fsType:overlay blockSize:0} overlay_0-262:{mountpoint:/var/lib/containers/storage/overlay/221d9929b57749f3a3c9144eb1f1e22a9369a6c478b49e1a950489d465368dc1/merged major:0 minor:262 fsType:overlay blockSize:0} overlay_0-268:{mountpoint:/var/lib/containers/storage/overlay/ab2271fae6f3b7259ba23617b8c37f8934bccc670696375a3b43e0f7bfc82e4f/merged major:0 minor:268 fsType:overlay blockSize:0} overlay_0-273:{mountpoint:/var/lib/containers/storage/overlay/09153c76b87077ee0818b2c8db7739094c4fc792e7ea91664c695bdbc1ed044c/merged major:0 minor:273 fsType:overlay blockSize:0} overlay_0-276:{mountpoint:/var/lib/containers/storage/overlay/2bb9bc22b54a7266721280e1071f5db438cb6a4a3f21b8eb26353fd099fb66a0/merged major:0 minor:276 fsType:overlay blockSize:0} overlay_0-295:{mountpoint:/var/lib/containers/storage/overlay/4284988fb707e995a3b04d9b45829310b3e307defec2ad54ebf46dffc7aba8ad/merged major:0 minor:295 fsType:overlay blockSize:0} overlay_0-300:{mountpoint:/var/lib/containers/storage/overlay/20d2c77c0b28f31e3e2046891b3d9696fb6f3b20e45734bf484c1bd5206da99b/merged major:0 minor:300 fsType:overlay blockSize:0} overlay_0-308:{mountpoint:/var/lib/containers/storage/overlay/adb468cab2396bba6f3408abab83e3c8434b72c7eea1152bb10bc993ec396794/merged major:0 minor:308 fsType:overlay blockSize:0} overlay_0-312:{mountpoint:/var/lib/containers/storage/overlay/48f5a98085eab83a15f87253b3a7619d8332f6a8bbe73c3f9d2d046775198940/merged major:0 minor:312 fsType:overlay blockSize:0} overlay_0-318:{mountpoint:/var/lib/containers/storage/overlay/0176b9ac6b7dc5c4425a859beec2cd230df4baacec09ab201a0867c726bde97e/merged major:0 minor:318 fsType:overlay blockSize:0} overlay_0-320:{mountpoint:/var/lib/containers/storage/overlay/33a8ff2e15586dad66d58de9919421b6269885f02b702498eaa94e37c2440fb5/merged major:0 minor:320 fsType:overlay blockSize:0} overlay_0-341:{mountpoint:/var/lib/containers/storage/overlay/93c9e4cb8e474b98754b6e447d28a907c1d7dc6aaafb68a9e0a42cb81a192805/merged major:0 minor:341 fsType:overlay blockSize:0} overlay_0-343:{mountpoint:/var/lib/containers/storage/overlay/f292dc1e3e16c063b05b71bb4558393103249b9eafdcb2ceb8b97e3549b7519f/merged major:0 minor:343 fsType:overlay blockSize:0} overlay_0-355:{mountpoint:/var/lib/containers/storage/overlay/1a5c07a827e385d04bcb2dc3d21fb4bcf2a756cb72147d6b97ab02ac1ebe1527/merged major:0 minor:355 fsType:overlay blockSize:0} overlay_0-357:{mountpoint:/var/lib/containers/storage/overlay/e54dd71c38d1851df42149a197e795e50f1d61da720b952a727a1afa5df34949/merged major:0 minor:357 fsType:overlay blockSize:0} overlay_0-359:{mountpoint:/var/lib/containers/storage/overlay/44778bcc6820dbb7d24afbbf8cea094d06ccd773b018731e7dfc43d9e62a0854/merged major:0 minor:359 fsType:overlay blockSize:0} overlay_0-364:{mountpoint:/var/lib/containers/storage/overlay/66ca1c49f161e655372e071c77e8c4ff755b46eb473e015d1625ed61134cda19/merged major:0 minor:364 fsType:overlay blockSize:0} overlay_0-372:{mountpoint:/var/lib/containers/storage/overlay/90b1cea6c71ff7de28946d2b911dc3824b4a4425ed1c68200cda9b714d3b08e4/merged major:0 minor:372 fsType:overlay blockSize:0} overlay_0-373:{mountpoint:/var/lib/containers/storage/overlay/96b9a25be60353a375841968618bc588533ec7acb3f4a6bfe2c840d51f1704b4/merged major:0 minor:373 fsType:overlay blockSize:0} overlay_0-378:{mountpoint:/var/lib/containers/storage/overlay/03e416254c94a03e47dc4149fc2d68aeb9d6521e4b764570bed0079d5d4bc899/merged major:0 minor:378 fsType:overlay blockSize:0} overlay_0-381:{mountpoint:/var/lib/containers/storage/overlay/da73d8533e0b7e093f69e1cff18bcbd06b1a84b5d0e8765d5e578c5fc18d38cc/merged major:0 minor:381 fsType:overlay blockSize:0} overlay_0-393:{mountpoint:/var/lib/containers/storage/overlay/01a431a3f217e5f0277eca0a8cf10886539caf1bf35bb97faae453e68d871bc8/merged major:0 minor:393 fsType:overlay blockSize:0} overlay_0-400:{mountpoint:/var/lib/containers/storage/overlay/3fbedb2411a1da4a1524d04f63417f0a7eb8940dd5b092943d6918549a3b768b/merged major:0 minor:400 fsType:overlay blockSize:0} overlay_0-405:{mountpoint:/var/lib/containers/storage/overlay/a26fe971f750061a3eed220e6d53dbf4566002d83c9455fa47080d0b1521174f/merged major:0 minor:405 fsType:overlay blockSize:0} overlay_0-410:{mountpoint:/var/lib/containers/storage/overlay/f7136b37426857539c6c6e0af330c959a86cd19c1c22110c95398dcea7617391/merged major:0 minor:410 fsType:overlay blockSize:0} overlay_0-416:{mountpoint:/var/lib/containers/storage/overlay/4928aea333d196e958bf3e0c290ff92971469073a426be2bf7e925d3dc08d54b/merged major:0 minor:416 fsType:overlay blockSize:0} overlay_0-422:{mountpoint:/var/lib/containers/storage/overlay/24a17ea3384c2be9f44d8037027ba8999eab9c528833a2acfdc821bfa092cdcc/merged major:0 minor:422 fsType:overlay blockSize:0} overlay_0-424:{mountpoint:/var/lib/containers/storage/overlay/0a3ab02cfde8d26934f72db6c5c9afe35a4d3542b8a935a2b48e7833ec879407/merged major:0 minor:424 fsType:overlay blockSize:0} overlay_0-429:{mountpoint:/var/lib/containers/storage/overlay/e7e46d2f224c02543e5416afae63c5467dd65364f1dd88040475739ae623d18c/merged major:0 minor:429 fsType:overlay blockSize:0} overlay_0-433:{mountpoint:/var/lib/containers/storage/overlay/8bc377622deb1455a917ff923c1bb9469814ea23ec6ad567d6c2288386905523/merged major:0 minor:433 fsType:overlay blockSize:0} overlay_0-438:{mountpoint:/var/lib/containers/storage/overlay/be9640872293e2f0e4653fe66563650f2418542856b9f2a60bcc62494ce1bf5f/merged major:0 minor:438 fsType:overlay blockSize:0} overlay_0-462:{mountpoint:/var/lib/containers/storage/overlay/3f1ca3424814a11e46068d2d8456bce545996dff381074ace14789eb8e35e310/merged major:0 minor:462 fsType:overlay blockSize:0} overlay_0-464:{mountpoint:/var/lib/containers/storage/overlay/b1d0b26e803769ebb4a5232a74afedbf87fa6afbe7865ac5e407f060d4ff2ca9/merged major:0 minor:464 fsType:overlay blockSize:0} overlay_0-466:{mountpoint:/var/lib/containers/storage/overlay/a08a99bc7bc32250a25b480cf0bd7f3642704107cea543d3459c7a9d0a49ff69/merged major:0 minor:466 fsType:overlay blockSize:0} overlay_0-472:{mountpoint:/var/lib/containers/storage/overlay/be4b3fee05bf7b7c2c176915f86f1aed2d5c3592093602526b2edba3c1b683c5/merged major:0 minor:472 fsType:overlay blockSize:0} overlay_0-477:{mountpoint:/var/lib/containers/storage/overlay/a6143b0b49aaa50779bed6fd66d95455f9dc6e692b7b76a2be33f95eeda1e9ae/merged major:0 minor:477 fsType:overlay blockSize:0} overlay_0-482:{mountpoint:/var/lib/containers/storage/overlay/7620cbf2f4be3c790f4bc2b3ee906156f9765eec2fad7f03ae5d350e721b20d4/merged major:0 minor:482 fsType:overlay blockSize:0} overlay_0-488:{mountpoint:/var/lib/containers/storage/overlay/e4bc9166d16f71dc14d10ba72a30b49b3943ab8d442af00929cbdee7b1485c41/merged major:0 minor:488 fsType:overlay blockSize:0} overlay_0-493:{mountpoint:/var/lib/containers/storage/overlay/cdea1299717fbf2fd981588f9f91100827551500aa3a8dfc597a73847fa9b594/merged major:0 minor:493 fsType:overlay blockSize:0} overlay_0-498:{mountpoint:/var/lib/containers/storage/overlay/1c19997ed8e6dc2cc78f2dc85eeb3c79f3bb80e341e0bfb8daf8712cce44fb91/merged major:0 minor:498 fsType:overlay blockSize:0} overlay_0-503:{mountpoint:/var/lib/containers/storage/overlay/d1198e408c81c043d1ea617dd8b3447384c2c5d19362ee412a7a50f8841589f6/merged major:0 minor:503 fsType:overlay blockSize:0} overlay_0-520:{mountpoint:/var/lib/containers/storage/overlay/72ae64432758b1ac74755683931d7883560ac42dda163122799072c814cfe9f7/merged major:0 minor:520 fsType:overlay blockSize:0} overlay_0-522:{mountpoint:/var/lib/containers/storage/overlay/c64c3db64ce82ea3c48b57f97bdd79a2cf61f0dd29fcfbeb54a53c1bd865bc51/merged major:0 minor:522 fsType:overlay blockSize:0} overlay_0-524:{mountpoint:/var/lib/containers/storage/overlay/fdebc2ba8f65acd36cfda867ba3f0802f717cd5b74a8fb60cd1a00e449efa793/merged major:0 minor:524 fsType:overlay blockSize:0} overlay_0-526:{mountpoint:/var/lib/containers/storage/overlay/e1208ada27fe1382ad5e6648e02d80cdf13464a1561997bca02ea636b6c8fbf4/merged major:0 minor:526 fsType:overlay blockSize:0} overlay_0-528:{mountpoint:/var/lib/containers/storage/overlay/03c5ef1d61fe7becbcf808dabcc47983a6e062752bc987492d4cbe6110fb3fd3/merged major:0 minor:528 fsType:overlay blockSize:0} overlay_0-534:{mountpoint:/var/lib/containers/storage/overlay/f4895c24769c12f45be6f32cbb7923e2d0df51f1e635ae940e74da17626e2859/merged major:0 minor:534 fsType:overlay blockSize:0} overlay_0-539:{mountpoint:/var/lib/containers/storage/overlay/ad6bb365bcd4df014a854e2648d37d9d20ae218f6af5c194a245510fcd9c04bb/merged major:0 minor:539 fsType:overlay blockSize:0} overlay_0-64:{mountpoint:/var/lib/containers/storage/overlay/97752c2e536cc6443cb0a2eccdd8e6022c9e66b5108f6e591a14c1608313dcb5/merged major:0 minor:64 fsType:overlay blockSize:0} overlay_0-66:{mountpoint:/var/lib/containers/storage/overlay/fceca0039a6ad0ddbde72a16dc07456d1da25e6c07311d77bb138bbdf82f9b31/merged major:0 minor:66 fsType:overlay blockSize:0} overlay_0-68:{mountpoint:/var/lib/containers/storage/overlay/de67130426dc846f1293d77dd94b993b3393a5946617a61a6420a12eb8aed13a/merged major:0 minor:68 fsType:overlay blockSize:0} overlay_0-72:{mountpoint:/var/lib/containers/storage/overlay/ed20e721053b41673af6be0f02fa328d1c3beb221fba42fd19d75c4059cee525/merged major:0 minor:72 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/29f9794ed0537c02dc386c4dfaba7191d38ccf82e21a6ea127105e3854e7618f/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/9ac77c3329e4ebbb0fb9db72a0773d6f39c021cfa77a81ece9c8b8bf2a74a034/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/e2922aa3c85d677589e0aab290e2a7ef2193db320ce388bb3f96c1c24b644ae6/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/fde815b5bac5fa6cc75e242a16f1f30a0181d76dbf41c62274cefaaa00eadae8/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/ffd07a7a190302cde5ed56da35bafb6ad4bf5f7e5bb9797cdc4ee93f0442eb2f/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/4596ebac1e76a9de8920f033d9a58ee00c367850effa3ac4a6be254d85d8a751/merged major:0 minor:96 fsType:overlay blockSize:0}] Oct 30 22:08:32.103583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.101858 5949 manager.go:217] Machine: {Timestamp:2026-10-30 22:08:32.101368426 +0000 UTC m=+0.065176988 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3076507 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e72500166de0eca2c19f9ae79a998 SystemUUID:ec2e7250-0166-de0e-ca2c-19f9ae79a998 BootID:f366a10e-7b18-4883-b9c8-d788d9a7e2ec Filesystems:[{Device:overlay_0-148 DeviceMajor:0 DeviceMinor:148 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/46b06955-7c3e-4a10-b5cd-5c2f4c34859f/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric DeviceMajor:0 DeviceMinor:285 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7062c9becc10226a3ca19a89d297a7269e2c51a6b6b7dca0f430f14dcd410149/userdata/shm DeviceMajor:0 DeviceMinor:353 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-393 DeviceMajor:0 DeviceMinor:393 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2e1024a48494975a37cd70c11675061811824847d6e8d357b5b6836a0f397c37/userdata/shm DeviceMajor:0 DeviceMinor:460 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8/volumes/kubernetes.io~projected/kube-api-access-hzvcp DeviceMajor:0 DeviceMinor:58 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/46b06955-7c3e-4a10-b5cd-5c2f4c34859f/volumes/kubernetes.io~projected/kube-api-access-bpcht DeviceMajor:0 DeviceMinor:288 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-534 DeviceMajor:0 DeviceMinor:534 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/5bb239d5-4f0e-4dd6-b58b-5f3b9e014895/volumes/kubernetes.io~projected/kube-api-access-knh4d DeviceMajor:0 DeviceMinor:56 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/46b06955-7c3e-4a10-b5cd-5c2f4c34859f/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:283 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9ac70a73-f7f0-4b3e-bf76-09869caefea0/volumes/kubernetes.io~projected/kube-api-access-bfsr2 DeviceMajor:0 DeviceMinor:348 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/fb3d3e48-51f2-4503-8026-79787ee98754/volumes/kubernetes.io~secret/telemeter-client-tls DeviceMajor:0 DeviceMinor:380 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/074eeae7-932c-4642-8fea-671898135ab1/volumes/kubernetes.io~projected/kube-api-access-8kxhz DeviceMajor:0 DeviceMinor:59 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/5dfaf077-3637-4cc9-8681-466c4e2cc5f4/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:203 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/df6b6cae-173c-461c-9195-4dce90bb6656/volumes/kubernetes.io~secret/kube-state-metrics-tls DeviceMajor:0 DeviceMinor:231 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/340578bb6c446168bf7e528fda076049b3664bf8c0ce11cb85be81554dd5d793/userdata/shm DeviceMajor:0 DeviceMinor:239 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-255 DeviceMajor:0 DeviceMinor:255 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-273 DeviceMajor:0 DeviceMinor:273 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-312 DeviceMajor:0 DeviceMinor:312 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-262 DeviceMajor:0 DeviceMinor:262 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4e612a431c19e72aa48209c41de2f70414134eda7dbfd4fe026be056f9641119/userdata/shm DeviceMajor:0 DeviceMinor:274 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/c373c767-d743-4819-be7c-4c9c830eb9db/volumes/kubernetes.io~projected/kube-api-access-cj545 DeviceMajor:0 DeviceMinor:202 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/df6b6cae-173c-461c-9195-4dce90bb6656/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:230 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/46b06955-7c3e-4a10-b5cd-5c2f4c34859f/volumes/kubernetes.io~secret/cluster-tls-config DeviceMajor:0 DeviceMinor:282 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/46b06955-7c3e-4a10-b5cd-5c2f4c34859f/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:286 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9c0d0893-e78b-48f3-bbe8-9f7d0381f42e/volumes/kubernetes.io~secret/monitoring-plugin-cert DeviceMajor:0 DeviceMinor:351 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-429 DeviceMajor:0 DeviceMinor:429 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-464 DeviceMajor:0 DeviceMinor:464 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-522 DeviceMajor:0 DeviceMinor:522 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-524 DeviceMajor:0 DeviceMinor:524 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4fbf0fd61fc6b065e8ca1ffc84f42505b9d8905160540d9402b8fc358977448c/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-138 DeviceMajor:0 DeviceMinor:138 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volume-subpaths/web-config/init-config-reloader/2 DeviceMajor:0 DeviceMinor:456 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0a6c5e5091712c18a370c9096a9488012885e7100be16362bfd46acebb21aee7/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a7b56859943b84b7b61cb86d28e1f5cb2bced2688531f8d082c00371d84af8f9/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4caf4e2411d03431dff003062637fa6d121bfffb6fc2c67e46afdbad23627e6a/userdata/shm DeviceMajor:0 DeviceMinor:237 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-245 DeviceMajor:0 DeviceMinor:245 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/46b06955-7c3e-4a10-b5cd-5c2f4c34859f/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy DeviceMajor:0 DeviceMinor:287 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/94283f0e-a719-4152-aea8-e219f7bb7eef/volumes/kubernetes.io~projected/kube-api-access-k2kkt DeviceMajor:0 DeviceMinor:307 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9ac70a73-f7f0-4b3e-bf76-09869caefea0/volumes/kubernetes.io~secret/client-ca-bundle DeviceMajor:0 DeviceMinor:347 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/46b06955-7c3e-4a10-b5cd-5c2f4c34859f/volume-subpaths/web-config/config-reloader/6 DeviceMajor:0 DeviceMinor:278 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/94283f0e-a719-4152-aea8-e219f7bb7eef/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:306 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/1b444016-092c-472b-979a-4a7b16271e27/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy DeviceMajor:0 DeviceMinor:329 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/1b444016-092c-472b-979a-4a7b16271e27/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:334 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/46b06955-7c3e-4a10-b5cd-5c2f4c34859f/volume-subpaths/web-config/alertmanager/9 DeviceMajor:0 DeviceMinor:278 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-364 DeviceMajor:0 DeviceMinor:364 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-416 DeviceMajor:0 DeviceMinor:416 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/68935e75-a4f0-4a54-9094-4f12b4a2dc39/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:234 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-243 DeviceMajor:0 DeviceMinor:243 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-341 DeviceMajor:0 DeviceMinor:341 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-378 DeviceMajor:0 DeviceMinor:378 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls DeviceMajor:0 DeviceMinor:451 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volumes/kubernetes.io~secret/secret-metrics-client-certs DeviceMajor:0 DeviceMinor:453 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:455 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-520 DeviceMajor:0 DeviceMinor:520 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:50 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44/volumes/kubernetes.io~projected/kube-api-access-2prlk DeviceMajor:0 DeviceMinor:54 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/89137e0ed84568ba01d3ac9b0e14fd3577d3f7a7d8a68c4b3d1a82cc43fc6601/userdata/shm DeviceMajor:0 DeviceMinor:70 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0ec135aec9bec53203b4fa8b33f029a49e2f2e4c576d202064835d2827d00338/userdata/shm DeviceMajor:0 DeviceMinor:74 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c373c767-d743-4819-be7c-4c9c830eb9db/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:45 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/c9381d26-e2af-4730-8b4e-06945c59585c/volumes/kubernetes.io~projected/kube-api-access-xf5c4 DeviceMajor:0 DeviceMinor:52 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/6b042dce-1fff-41a2-83dd-ae18f3098207/volumes/kubernetes.io~projected/kube-api-access-qxdbc DeviceMajor:0 DeviceMinor:55 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/62773096baa81d57351c85102abff61a5273833dd16c64c61a2dbbc4c56411dd/userdata/shm DeviceMajor:0 DeviceMinor:241 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/46b06955-7c3e-4a10-b5cd-5c2f4c34859f/volumes/kubernetes.io~secret/secret-alertmanager-main-tls DeviceMajor:0 DeviceMinor:284 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-343 DeviceMajor:0 DeviceMinor:343 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/46b06955-7c3e-4a10-b5cd-5c2f4c34859f/volume-subpaths/web-config/init-config-reloader/6 DeviceMajor:0 DeviceMinor:278 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-405 DeviceMajor:0 DeviceMinor:405 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls DeviceMajor:0 DeviceMinor:440 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/074eeae7-932c-4642-8fea-671898135ab1/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:49 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/62c19734cc22ee07d6a2c049f1ee538a83cb1a6158f7b1dde8efdf1c7f7767f6/userdata/shm DeviceMajor:0 DeviceMinor:209 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-300 DeviceMajor:0 DeviceMinor:300 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3155dfa9-0dc2-4256-9e16-490b3721702d/volumes/kubernetes.io~projected/kube-api-access-2gvkg DeviceMajor:0 DeviceMinor:513 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-528 DeviceMajor:0 DeviceMinor:528 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/run/containers/storage/overlay-containers/36051ba353f4d1493e47afd3bee1fa39336698eac55c1ddd7a97c40dbd52bdfa/userdata/shm DeviceMajor:0 DeviceMinor:61 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-64 DeviceMajor:0 DeviceMinor:64 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/976467f0e8d8d1bfa25ef9bd6e507f4786388b419da2bcea0765ec9a620aba24/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-318 DeviceMajor:0 DeviceMinor:318 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fb3d3e48-51f2-4503-8026-79787ee98754/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:391 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-422 DeviceMajor:0 DeviceMinor:422 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:450 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-72 DeviceMajor:0 DeviceMinor:72 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/eb20dbc00c4333ecb219b11f019b76d9cd02e2de6867ca00a83081ff0adc9951/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-147 DeviceMajor:0 DeviceMinor:147 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-247 DeviceMajor:0 DeviceMinor:247 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fb3d3e48-51f2-4503-8026-79787ee98754/volumes/kubernetes.io~secret/federate-client-tls DeviceMajor:0 DeviceMinor:390 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-493 DeviceMajor:0 DeviceMinor:493 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f70f76b15ecaf9d5fff82369559697fc0109804c1f08a91ec6e029cb3acea159/userdata/shm DeviceMajor:0 DeviceMinor:515 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-539 DeviceMajor:0 DeviceMinor:539 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e5779bc8-21ad-465e-8749-8f7e9afdc03f/volumes/kubernetes.io~projected/kube-api-access-h7f59 DeviceMajor:0 DeviceMinor:53 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/46b06955-7c3e-4a10-b5cd-5c2f4c34859f/volumes/kubernetes.io~secret/config-volume DeviceMajor:0 DeviceMinor:279 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/1b444016-092c-472b-979a-4a7b16271e27/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules DeviceMajor:0 DeviceMinor:331 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a432ddbd05d471d3613077a1a33b72f9dc711e4b0edb4221802c3fff94c648b8/userdata/shm DeviceMajor:0 DeviceMinor:335 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9ac70a73-f7f0-4b3e-bf76-09869caefea0/volumes/kubernetes.io~secret/secret-metrics-server-tls DeviceMajor:0 DeviceMinor:345 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/61a90835359d781127b7c1b0c8148cf6aef8e3c7ad74148743763ac42eb9b7ad/userdata/shm DeviceMajor:0 DeviceMinor:60 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-68 DeviceMajor:0 DeviceMinor:68 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-219 DeviceMajor:0 DeviceMinor:219 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1b444016-092c-472b-979a-4a7b16271e27/volumes/kubernetes.io~projected/kube-api-access-j9862 DeviceMajor:0 DeviceMinor:336 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9c0d0893-e78b-48f3-bbe8-9f7d0381f42e/volumes/kubernetes.io~projected/kube-api-access-hxr72 DeviceMajor:0 DeviceMinor:352 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-355 DeviceMajor:0 DeviceMinor:355 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-462 DeviceMajor:0 DeviceMinor:462 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/5dfaf077-3637-4cc9-8681-466c4e2cc5f4/volumes/kubernetes.io~projected/kube-api-access-8r24v DeviceMajor:0 DeviceMinor:204 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/46b06955-7c3e-4a10-b5cd-5c2f4c34859f/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:278 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-381 DeviceMajor:0 DeviceMinor:381 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:457 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volume-subpaths/web-config/prometheus/15 DeviceMajor:0 DeviceMinor:456 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volume-subpaths/web-config/config-reloader/2 DeviceMajor:0 DeviceMinor:456 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/65c2ef45-7211-4828-86bc-c9d7af39ec66/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:508 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-295 DeviceMajor:0 DeviceMinor:295 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-503 DeviceMajor:0 DeviceMinor:503 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d71deada5e7ff6a92740256adf5084afe01473c75b9427829a90d8b0c608cd3a/userdata/shm DeviceMajor:0 DeviceMinor:518 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-526 DeviceMajor:0 DeviceMinor:526 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-276 DeviceMajor:0 DeviceMinor:276 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-151 DeviceMajor:0 DeviceMinor:151 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/11464051-0aa4-4324-b2c7-7ea2096bbed6/volumes/kubernetes.io~projected/kube-api-access-nrwhs DeviceMajor:0 DeviceMinor:51 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/1b444016-092c-472b-979a-4a7b16271e27/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:333 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-372 DeviceMajor:0 DeviceMinor:372 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fb3d3e48-51f2-4503-8026-79787ee98754/volumes/kubernetes.io~projected/kube-api-access-jtbtv DeviceMajor:0 DeviceMinor:392 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-477 DeviceMajor:0 DeviceMinor:477 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-66 DeviceMajor:0 DeviceMinor:66 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/df6b6cae-173c-461c-9195-4dce90bb6656/volumes/kubernetes.io~projected/kube-api-access-45l5v DeviceMajor:0 DeviceMinor:232 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/46b06955-7c3e-4a10-b5cd-5c2f4c34859f/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:281 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-268 DeviceMajor:0 DeviceMinor:268 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fb3d3e48-51f2-4503-8026-79787ee98754/volumes/kubernetes.io~secret/secret-telemeter-client DeviceMajor:0 DeviceMinor:383 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-410 DeviceMajor:0 DeviceMinor:410 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-482 DeviceMajor:0 DeviceMinor:482 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-488 DeviceMajor:0 DeviceMinor:488 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/50767516a7f3953ef0ef84dec6c641854ecd806e5f5a3b2c0f75d7e46408cc12/userdata/shm DeviceMajor:0 DeviceMinor:207 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e129700d-0583-4316-bcee-431c57b0effa/volumes/kubernetes.io~projected/kube-api-access-5zxs5 DeviceMajor:0 DeviceMinor:233 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/68935e75-a4f0-4a54-9094-4f12b4a2dc39/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:235 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/cd91dfa17aa2f5dbb8bf2c7c3f923a026fe1805f799b2dd62fe87019e250f503/userdata/shm DeviceMajor:0 DeviceMinor:398 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-424 DeviceMajor:0 DeviceMinor:424 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:454 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/65c2ef45-7211-4828-86bc-c9d7af39ec66/volumes/kubernetes.io~projected/kube-api-access-dpz88 DeviceMajor:0 DeviceMinor:57 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f64980b7-13c0-40c3-aa3f-277916f24518/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:197 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-257 DeviceMajor:0 DeviceMinor:257 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1b444016-092c-472b-979a-4a7b16271e27/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics DeviceMajor:0 DeviceMinor:332 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-466 DeviceMajor:0 DeviceMinor:466 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-498 DeviceMajor:0 DeviceMinor:498 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f64980b7-13c0-40c3-aa3f-277916f24518/volumes/kubernetes.io~projected/kube-api-access-tjk5b DeviceMajor:0 DeviceMinor:201 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-213 DeviceMajor:0 DeviceMinor:213 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-400 DeviceMajor:0 DeviceMinor:400 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/94283f0e-a719-4152-aea8-e219f7bb7eef/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:305 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2320e8be4a7af79dbed9ac74a3043708a1694e6a3e6ecd5a21a60e433f61e3b5/userdata/shm DeviceMajor:0 DeviceMinor:310 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-320 DeviceMajor:0 DeviceMinor:320 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-357 DeviceMajor:0 DeviceMinor:357 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-359 DeviceMajor:0 DeviceMinor:359 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:456 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/074eeae7-932c-4642-8fea-671898135ab1/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-308 DeviceMajor:0 DeviceMinor:308 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/46b06955-7c3e-4a10-b5cd-5c2f4c34859f/volume-subpaths/cluster-tls-config/alertmanager/10 DeviceMajor:0 DeviceMinor:282 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-373 DeviceMajor:0 DeviceMinor:373 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file DeviceMajor:0 DeviceMinor:452 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-472 DeviceMajor:0 DeviceMinor:472 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e77edf5e5faa5574836fffe4f9ac84164a97e19598785877baa1d628cf1b0a58/userdata/shm DeviceMajor:0 DeviceMinor:205 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/68935e75-a4f0-4a54-9094-4f12b4a2dc39/volumes/kubernetes.io~projected/kube-api-access-z6fv5 DeviceMajor:0 DeviceMinor:236 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9ac70a73-f7f0-4b3e-bf76-09869caefea0/volumes/kubernetes.io~secret/secret-metrics-server-client-certs DeviceMajor:0 DeviceMinor:346 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-438 DeviceMajor:0 DeviceMinor:438 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volumes/kubernetes.io~secret/config DeviceMajor:0 DeviceMinor:441 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/cd46f3e8-c752-4eed-a65d-07e060bf32b4/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:512 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/f6cc017b-3f50-423a-93da-e4515336a2bc/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:47 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/81b300842b95d3636a9caf7f46ce2fbcf3b083baa3f13105be77089783c54aeb/userdata/shm DeviceMajor:0 DeviceMinor:78 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e129700d-0583-4316-bcee-431c57b0effa/volumes/kubernetes.io~secret/openshift-state-metrics-tls DeviceMajor:0 DeviceMinor:229 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1f0618e251474d6ffb7acd4a8504e72915215cd2c4d8e67c1e553cced27ae6d2/userdata/shm DeviceMajor:0 DeviceMinor:349 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-433 DeviceMajor:0 DeviceMinor:433 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:48 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-217 DeviceMajor:0 DeviceMinor:217 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e129700d-0583-4316-bcee-431c57b0effa/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:225 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-249 DeviceMajor:0 DeviceMinor:249 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1b444016-092c-472b-979a-4a7b16271e27/volumes/kubernetes.io~secret/secret-thanos-querier-tls DeviceMajor:0 DeviceMinor:330 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volumes/kubernetes.io~secret/secret-kube-rbac-proxy DeviceMajor:0 DeviceMinor:458 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volumes/kubernetes.io~projected/kube-api-access-qsvp8 DeviceMajor:0 DeviceMinor:459 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1b3456bb2efcc685f0f73cec86542ed7117b7502a4cba412c802ad4c9d644ed6/userdata/shm DeviceMajor:0 DeviceMinor:514 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:1b3456bb2efcc68 MacAddress:1e:17:1d:fb:34:9b Speed:10000 Mtu:8901} {Name:1f0618e251474d6 MacAddress:c6:b1:ff:b6:51:a2 Speed:10000 Mtu:8901} {Name:2320e8be4a7af79 MacAddress:8a:e1:31:46:a1:9e Speed:10000 Mtu:8901} {Name:2e1024a48494975 MacAddress:e2:47:ab:45:10:0f Speed:10000 Mtu:8901} {Name:340578bb6c44616 MacAddress:56:54:3f:5f:b5:50 Speed:10000 Mtu:8901} {Name:4caf4e2411d0343 MacAddress:1e:95:c7:70:7d:42 Speed:10000 Mtu:8901} {Name:4e612a431c19e72 MacAddress:26:74:bd:8e:4b:a0 Speed:10000 Mtu:8901} {Name:50767516a7f3953 MacAddress:16:06:22:9c:fd:04 Speed:10000 Mtu:8901} {Name:62c19734cc22ee0 MacAddress:ae:15:03:97:36:23 Speed:10000 Mtu:8901} {Name:7062c9becc10226 MacAddress:c6:1c:ec:63:e8:29 Speed:10000 Mtu:8901} {Name:a432ddbd05d471d MacAddress:26:ea:98:b1:ff:2d Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:f6:06:9a:19 Speed:0 Mtu:9001} {Name:br-int MacAddress:be:76:1a:96:9a:c1 Speed:0 Mtu:8901} {Name:cd91dfa17aa2f5d MacAddress:be:ac:91:29:78:38 Speed:10000 Mtu:8901} {Name:d71deada5e7ff6a MacAddress:7e:9e:76:be:25:56 Speed:10000 Mtu:8901} {Name:e77edf5e5faa557 MacAddress:12:c6:c0:34:07:06 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:f6:06:9a:19 Speed:0 Mtu:9001} {Name:f70f76b15ecaf9d MacAddress:e6:ba:c7:8a:1d:3f Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:8e:81:42:5c:87:16 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:86:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:ee:3d:6f:e9:85:16 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Oct 30 22:08:32.103583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.102339 5949 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Oct 30 22:08:32.103583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.102384 5949 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Oct 30 22:08:32.103583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.102677 5949 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Oct 30 22:08:32.103583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.102705 5949 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-24.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Oct 30 22:08:32.103583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.102828 5949 topology_manager.go:143] "Creating topology manager with none policy" Oct 30 22:08:32.103583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.102836 5949 container_manager_linux.go:311] "Creating device plugin manager" Oct 30 22:08:32.103583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.102846 5949 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 22:08:32.103583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.102865 5949 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 22:08:32.103583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.102941 5949 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Oct 30 22:08:32.103583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.103089 5949 state_mem.go:41] "Initialized" logger="CPUManager state memory" Oct 30 22:08:32.103583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.103202 5949 server.go:1235] "Using root directory" path="/var/lib/kubelet" Oct 30 22:08:32.103583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.103262 5949 kubelet.go:493] "Attempting to sync node with API server" Oct 30 22:08:32.103583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.103276 5949 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Oct 30 22:08:32.103583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.103285 5949 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Oct 30 22:08:32.103583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.103293 5949 kubelet.go:405] "Adding apiserver pod source" Oct 30 22:08:32.103583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.103302 5949 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Oct 30 22:08:32.104409 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.104272 5949 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Oct 30 22:08:32.104409 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.104284 5949 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 22:08:32.104766 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.104744 5949 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Oct 30 22:08:32.107984 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.107968 5949 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Oct 30 22:08:32.107984 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.107982 5949 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 22:08:32.108121 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.108093 5949 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem" Oct 30 22:08:32.108305 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.108293 5949 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Oct 30 22:08:32.108343 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.108323 5949 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Oct 30 22:08:32.108437 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.108427 5949 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Oct 30 22:08:32.108467 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.108448 5949 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Oct 30 22:08:32.108467 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.108456 5949 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Oct 30 22:08:32.108467 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.108463 5949 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Oct 30 22:08:32.108552 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.108471 5949 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Oct 30 22:08:32.108552 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.108477 5949 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Oct 30 22:08:32.108552 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.108483 5949 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Oct 30 22:08:32.108552 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.108488 5949 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Oct 30 22:08:32.108552 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.108502 5949 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Oct 30 22:08:32.108552 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.108519 5949 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Oct 30 22:08:32.108552 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.108531 5949 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Oct 30 22:08:32.108552 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.108540 5949 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Oct 30 22:08:32.108768 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.108583 5949 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Oct 30 22:08:32.108768 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.108589 5949 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Oct 30 22:08:32.111984 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.111969 5949 server.go:1263] "Started kubelet" Oct 30 22:08:32.112883 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.112075 5949 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Oct 30 22:08:32.112700 ip-10-0-142-24 systemd[1]: Started Kubernetes Kubelet. Oct 30 22:08:32.117027 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.115034 5949 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Oct 30 22:08:32.117027 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.115478 5949 server_v1.go:49] "podresources" method="list" useActivePods=true Oct 30 22:08:32.117027 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.115853 5949 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Oct 30 22:08:32.117027 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.116236 5949 server.go:317] "Adding debug handlers to kubelet server" Oct 30 22:08:32.120550 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.120530 5949 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Oct 30 22:08:32.120666 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.120645 5949 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Oct 30 22:08:32.120723 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:32.120668 5949 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-24.ec2.internal\" not found" Oct 30 22:08:32.120768 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.120736 5949 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 22:02:22 +0000 UTC" deadline="2028-06-21 03:45:43.919058832 +0000 UTC" Oct 30 22:08:32.120768 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.120752 5949 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14381h37m11.79830916s" Oct 30 22:08:32.121273 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.120806 5949 volume_manager.go:309] "The desired_state_of_world populator starts" Oct 30 22:08:32.121273 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.121258 5949 volume_manager.go:311] "Starting Kubelet Volume Manager" Oct 30 22:08:32.121428 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.120936 5949 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Oct 30 22:08:32.121521 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.121504 5949 factory.go:55] Registering systemd factory Oct 30 22:08:32.121568 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.121526 5949 factory.go:223] Registration of the systemd container factory successfully Oct 30 22:08:32.122140 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.122122 5949 factory.go:153] Registering CRI-O factory Oct 30 22:08:32.122140 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.122141 5949 factory.go:223] Registration of the crio container factory successfully Oct 30 22:08:32.122294 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.122188 5949 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Oct 30 22:08:32.122294 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.122207 5949 factory.go:103] Registering Raw factory Oct 30 22:08:32.122294 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.122236 5949 manager.go:1199] Started watching for new ooms in manager Oct 30 22:08:32.122548 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.122534 5949 manager.go:319] Starting recovery of all containers Oct 30 22:08:32.125316 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125264 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="074eeae7-932c-4642-8fea-671898135ab1" volumeName="kubernetes.io/configmap/074eeae7-932c-4642-8fea-671898135ab1-env-overrides" seLinuxMountContext="" Oct 30 22:08:32.125402 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125319 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3913a2c9-e88c-4f49-bc63-02b1fa543c38" volumeName="kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-prometheus-k8s-tls" seLinuxMountContext="" Oct 30 22:08:32.125402 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125343 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9ac70a73-f7f0-4b3e-bf76-09869caefea0" volumeName="kubernetes.io/empty-dir/9ac70a73-f7f0-4b3e-bf76-09869caefea0-audit-log" seLinuxMountContext="" Oct 30 22:08:32.125402 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125358 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9c0d0893-e78b-48f3-bbe8-9f7d0381f42e" volumeName="kubernetes.io/projected/9c0d0893-e78b-48f3-bbe8-9f7d0381f42e-kube-api-access-hxr72" seLinuxMountContext="" Oct 30 22:08:32.125402 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125371 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb3d3e48-51f2-4503-8026-79787ee98754" volumeName="kubernetes.io/configmap/fb3d3e48-51f2-4503-8026-79787ee98754-telemeter-trusted-ca-bundle" seLinuxMountContext="" Oct 30 22:08:32.125402 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125384 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="074eeae7-932c-4642-8fea-671898135ab1" volumeName="kubernetes.io/projected/074eeae7-932c-4642-8fea-671898135ab1-kube-api-access-8kxhz" seLinuxMountContext="" Oct 30 22:08:32.125402 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125396 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1b444016-092c-472b-979a-4a7b16271e27" volumeName="kubernetes.io/secret/1b444016-092c-472b-979a-4a7b16271e27-secret-grpc-tls" seLinuxMountContext="" Oct 30 22:08:32.125696 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125409 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3913a2c9-e88c-4f49-bc63-02b1fa543c38" volumeName="kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-k8s-rulefiles-1" seLinuxMountContext="" Oct 30 22:08:32.125696 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125428 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" volumeName="kubernetes.io/projected/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-tls-assets" seLinuxMountContext="" Oct 30 22:08:32.125696 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125444 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" volumeName="kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-cluster-tls-config" seLinuxMountContext="" Oct 30 22:08:32.125696 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125469 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5bb239d5-4f0e-4dd6-b58b-5f3b9e014895" volumeName="kubernetes.io/projected/5bb239d5-4f0e-4dd6-b58b-5f3b9e014895-kube-api-access-knh4d" seLinuxMountContext="" Oct 30 22:08:32.125696 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125483 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6b042dce-1fff-41a2-83dd-ae18f3098207" volumeName="kubernetes.io/empty-dir/6b042dce-1fff-41a2-83dd-ae18f3098207-tmp-dir" seLinuxMountContext="" Oct 30 22:08:32.125696 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125494 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="94283f0e-a719-4152-aea8-e219f7bb7eef" volumeName="kubernetes.io/empty-dir/94283f0e-a719-4152-aea8-e219f7bb7eef-tmp" seLinuxMountContext="" Oct 30 22:08:32.125696 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125509 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="94283f0e-a719-4152-aea8-e219f7bb7eef" volumeName="kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-serving-cert" seLinuxMountContext="" Oct 30 22:08:32.125696 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125545 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3913a2c9-e88c-4f49-bc63-02b1fa543c38" volumeName="kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-k8s-rulefiles-0" seLinuxMountContext="" Oct 30 22:08:32.125696 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125558 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3913a2c9-e88c-4f49-bc63-02b1fa543c38" volumeName="kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-config" seLinuxMountContext="" Oct 30 22:08:32.125696 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125585 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9ac70a73-f7f0-4b3e-bf76-09869caefea0" volumeName="kubernetes.io/configmap/9ac70a73-f7f0-4b3e-bf76-09869caefea0-configmap-kubelet-serving-ca-bundle" seLinuxMountContext="" Oct 30 22:08:32.125696 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125598 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9ac70a73-f7f0-4b3e-bf76-09869caefea0" volumeName="kubernetes.io/secret/9ac70a73-f7f0-4b3e-bf76-09869caefea0-secret-metrics-server-tls" seLinuxMountContext="" Oct 30 22:08:32.125696 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125609 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c9381d26-e2af-4730-8b4e-06945c59585c" volumeName="kubernetes.io/configmap/c9381d26-e2af-4730-8b4e-06945c59585c-cni-binary-copy" seLinuxMountContext="" Oct 30 22:08:32.125696 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125621 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="074eeae7-932c-4642-8fea-671898135ab1" volumeName="kubernetes.io/configmap/074eeae7-932c-4642-8fea-671898135ab1-ovnkube-script-lib" seLinuxMountContext="" Oct 30 22:08:32.125696 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125635 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1b444016-092c-472b-979a-4a7b16271e27" volumeName="kubernetes.io/projected/1b444016-092c-472b-979a-4a7b16271e27-kube-api-access-j9862" seLinuxMountContext="" Oct 30 22:08:32.125696 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125650 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5dfaf077-3637-4cc9-8681-466c4e2cc5f4" volumeName="kubernetes.io/projected/5dfaf077-3637-4cc9-8681-466c4e2cc5f4-kube-api-access-8r24v" seLinuxMountContext="" Oct 30 22:08:32.125696 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125663 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="68935e75-a4f0-4a54-9094-4f12b4a2dc39" volumeName="kubernetes.io/secret/68935e75-a4f0-4a54-9094-4f12b4a2dc39-node-exporter-kube-rbac-proxy-config" seLinuxMountContext="" Oct 30 22:08:32.125696 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125675 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e129700d-0583-4316-bcee-431c57b0effa" volumeName="kubernetes.io/projected/e129700d-0583-4316-bcee-431c57b0effa-kube-api-access-5zxs5" seLinuxMountContext="" Oct 30 22:08:32.125696 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125690 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb3d3e48-51f2-4503-8026-79787ee98754" volumeName="kubernetes.io/secret/fb3d3e48-51f2-4503-8026-79787ee98754-secret-telemeter-client" seLinuxMountContext="" Oct 30 22:08:32.125696 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125703 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" volumeName="kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-secret-alertmanager-kube-rbac-proxy-web" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125716 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="df6b6cae-173c-461c-9195-4dce90bb6656" volumeName="kubernetes.io/projected/df6b6cae-173c-461c-9195-4dce90bb6656-kube-api-access-45l5v" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125730 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f6cc017b-3f50-423a-93da-e4515336a2bc" volumeName="kubernetes.io/configmap/f6cc017b-3f50-423a-93da-e4515336a2bc-konnectivity-ca" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125748 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="65c2ef45-7211-4828-86bc-c9d7af39ec66" volumeName="kubernetes.io/secret/65c2ef45-7211-4828-86bc-c9d7af39ec66-metrics-certs" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125760 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9c0d0893-e78b-48f3-bbe8-9f7d0381f42e" volumeName="kubernetes.io/secret/9c0d0893-e78b-48f3-bbe8-9f7d0381f42e-monitoring-plugin-cert" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125774 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44" volumeName="kubernetes.io/empty-dir/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-tmp" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125786 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="65c2ef45-7211-4828-86bc-c9d7af39ec66" volumeName="kubernetes.io/projected/65c2ef45-7211-4828-86bc-c9d7af39ec66-kube-api-access-dpz88" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125799 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6b042dce-1fff-41a2-83dd-ae18f3098207" volumeName="kubernetes.io/projected/6b042dce-1fff-41a2-83dd-ae18f3098207-kube-api-access-qxdbc" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125836 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="df6b6cae-173c-461c-9195-4dce90bb6656" volumeName="kubernetes.io/configmap/df6b6cae-173c-461c-9195-4dce90bb6656-kube-state-metrics-custom-resource-state-configmap" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125849 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1b444016-092c-472b-979a-4a7b16271e27" volumeName="kubernetes.io/secret/1b444016-092c-472b-979a-4a7b16271e27-secret-thanos-querier-kube-rbac-proxy" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125862 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="df6b6cae-173c-461c-9195-4dce90bb6656" volumeName="kubernetes.io/configmap/df6b6cae-173c-461c-9195-4dce90bb6656-metrics-client-ca" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125873 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="df6b6cae-173c-461c-9195-4dce90bb6656" volumeName="kubernetes.io/secret/df6b6cae-173c-461c-9195-4dce90bb6656-kube-state-metrics-kube-rbac-proxy-config" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125886 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb3d3e48-51f2-4503-8026-79787ee98754" volumeName="kubernetes.io/configmap/fb3d3e48-51f2-4503-8026-79787ee98754-serving-certs-ca-bundle" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125901 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="074eeae7-932c-4642-8fea-671898135ab1" volumeName="kubernetes.io/configmap/074eeae7-932c-4642-8fea-671898135ab1-ovnkube-config" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125913 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3913a2c9-e88c-4f49-bc63-02b1fa543c38" volumeName="kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-configmap-serving-certs-ca-bundle" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125926 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3913a2c9-e88c-4f49-bc63-02b1fa543c38" volumeName="kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-metrics-client-certs" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125938 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="94283f0e-a719-4152-aea8-e219f7bb7eef" volumeName="kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-oauth-serving-cert" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125950 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c1f7e385-c1ea-4fd9-aff3-513ae337b5d8" volumeName="kubernetes.io/configmap/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-multus-daemon-config" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125962 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c373c767-d743-4819-be7c-4c9c830eb9db" volumeName="kubernetes.io/empty-dir/c373c767-d743-4819-be7c-4c9c830eb9db-tmp-dir" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125974 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e129700d-0583-4316-bcee-431c57b0effa" volumeName="kubernetes.io/secret/e129700d-0583-4316-bcee-431c57b0effa-openshift-state-metrics-kube-rbac-proxy-config" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.125989 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e5779bc8-21ad-465e-8749-8f7e9afdc03f" volumeName="kubernetes.io/projected/e5779bc8-21ad-465e-8749-8f7e9afdc03f-kube-api-access-h7f59" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126002 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3913a2c9-e88c-4f49-bc63-02b1fa543c38" volumeName="kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-grpc-tls" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126015 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3913a2c9-e88c-4f49-bc63-02b1fa543c38" volumeName="kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-web-config" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126029 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5dfaf077-3637-4cc9-8681-466c4e2cc5f4" volumeName="kubernetes.io/secret/5dfaf077-3637-4cc9-8681-466c4e2cc5f4-insights-runtime-extractor-tls" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126043 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb3d3e48-51f2-4503-8026-79787ee98754" volumeName="kubernetes.io/configmap/fb3d3e48-51f2-4503-8026-79787ee98754-metrics-client-ca" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126057 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3913a2c9-e88c-4f49-bc63-02b1fa543c38" volumeName="kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-kube-rbac-proxy" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126069 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" volumeName="kubernetes.io/configmap/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-metrics-client-ca" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126083 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5dfaf077-3637-4cc9-8681-466c4e2cc5f4" volumeName="kubernetes.io/configmap/5dfaf077-3637-4cc9-8681-466c4e2cc5f4-kube-rbac-proxy-cm" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126097 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c1f7e385-c1ea-4fd9-aff3-513ae337b5d8" volumeName="kubernetes.io/configmap/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-cni-binary-copy" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126110 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e129700d-0583-4316-bcee-431c57b0effa" volumeName="kubernetes.io/secret/e129700d-0583-4316-bcee-431c57b0effa-openshift-state-metrics-tls" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126122 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f64980b7-13c0-40c3-aa3f-277916f24518" volumeName="kubernetes.io/secret/f64980b7-13c0-40c3-aa3f-277916f24518-cert" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126146 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1b444016-092c-472b-979a-4a7b16271e27" volumeName="kubernetes.io/empty-dir/1b444016-092c-472b-979a-4a7b16271e27-tmp" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126158 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" volumeName="kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-secret-alertmanager-kube-rbac-proxy-metric" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126172 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="68935e75-a4f0-4a54-9094-4f12b4a2dc39" volumeName="kubernetes.io/configmap/68935e75-a4f0-4a54-9094-4f12b4a2dc39-node-exporter-accelerators-collector-config" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126185 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1b444016-092c-472b-979a-4a7b16271e27" volumeName="kubernetes.io/secret/1b444016-092c-472b-979a-4a7b16271e27-secret-thanos-querier-tls" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126200 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3913a2c9-e88c-4f49-bc63-02b1fa543c38" volumeName="kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-k8s-rulefiles-2" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126212 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c1f7e385-c1ea-4fd9-aff3-513ae337b5d8" volumeName="kubernetes.io/projected/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-kube-api-access-hzvcp" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126240 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c373c767-d743-4819-be7c-4c9c830eb9db" volumeName="kubernetes.io/secret/c373c767-d743-4819-be7c-4c9c830eb9db-metrics-tls" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126253 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cd46f3e8-c752-4eed-a65d-07e060bf32b4" volumeName="kubernetes.io/secret/cd46f3e8-c752-4eed-a65d-07e060bf32b4-original-pull-secret" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126266 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="074eeae7-932c-4642-8fea-671898135ab1" volumeName="kubernetes.io/secret/074eeae7-932c-4642-8fea-671898135ab1-ovn-node-metrics-cert" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126281 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3913a2c9-e88c-4f49-bc63-02b1fa543c38" volumeName="kubernetes.io/empty-dir/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-k8s-db" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126295 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3913a2c9-e88c-4f49-bc63-02b1fa543c38" volumeName="kubernetes.io/projected/3913a2c9-e88c-4f49-bc63-02b1fa543c38-kube-api-access-qsvp8" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126307 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3913a2c9-e88c-4f49-bc63-02b1fa543c38" volumeName="kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-prometheus-k8s-thanos-sidecar-tls" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126320 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44" volumeName="kubernetes.io/projected/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-kube-api-access-2prlk" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126334 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5dfaf077-3637-4cc9-8681-466c4e2cc5f4" volumeName="kubernetes.io/empty-dir/5dfaf077-3637-4cc9-8681-466c4e2cc5f4-data-volume" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126346 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="94283f0e-a719-4152-aea8-e219f7bb7eef" volumeName="kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-console-config" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126358 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9ac70a73-f7f0-4b3e-bf76-09869caefea0" volumeName="kubernetes.io/configmap/9ac70a73-f7f0-4b3e-bf76-09869caefea0-metrics-server-audit-profiles" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126370 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9ac70a73-f7f0-4b3e-bf76-09869caefea0" volumeName="kubernetes.io/secret/9ac70a73-f7f0-4b3e-bf76-09869caefea0-client-ca-bundle" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126381 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb3d3e48-51f2-4503-8026-79787ee98754" volumeName="kubernetes.io/projected/fb3d3e48-51f2-4503-8026-79787ee98754-kube-api-access-jtbtv" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126394 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb3d3e48-51f2-4503-8026-79787ee98754" volumeName="kubernetes.io/secret/fb3d3e48-51f2-4503-8026-79787ee98754-federate-client-tls" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126406 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1b444016-092c-472b-979a-4a7b16271e27" volumeName="kubernetes.io/configmap/1b444016-092c-472b-979a-4a7b16271e27-metrics-client-ca" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126420 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" volumeName="kubernetes.io/empty-dir/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-config-out" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126437 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5bb239d5-4f0e-4dd6-b58b-5f3b9e014895" volumeName="kubernetes.io/configmap/5bb239d5-4f0e-4dd6-b58b-5f3b9e014895-iptables-alerter-script" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126452 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c9381d26-e2af-4730-8b4e-06945c59585c" volumeName="kubernetes.io/projected/c9381d26-e2af-4730-8b4e-06945c59585c-kube-api-access-xf5c4" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126464 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="df6b6cae-173c-461c-9195-4dce90bb6656" volumeName="kubernetes.io/empty-dir/df6b6cae-173c-461c-9195-4dce90bb6656-volume-directive-shadow" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126509 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb3d3e48-51f2-4503-8026-79787ee98754" volumeName="kubernetes.io/secret/fb3d3e48-51f2-4503-8026-79787ee98754-telemeter-client-tls" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126525 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb3d3e48-51f2-4503-8026-79787ee98754" volumeName="kubernetes.io/secret/fb3d3e48-51f2-4503-8026-79787ee98754-secret-telemeter-client-kube-rbac-proxy-config" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126559 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3913a2c9-e88c-4f49-bc63-02b1fa543c38" volumeName="kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-configmap-kubelet-serving-ca-bundle" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126572 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" volumeName="kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-secret-alertmanager-main-tls" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126598 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9ac70a73-f7f0-4b3e-bf76-09869caefea0" volumeName="kubernetes.io/projected/9ac70a73-f7f0-4b3e-bf76-09869caefea0-kube-api-access-bfsr2" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126605 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9ac70a73-f7f0-4b3e-bf76-09869caefea0" volumeName="kubernetes.io/secret/9ac70a73-f7f0-4b3e-bf76-09869caefea0-secret-metrics-server-client-certs" seLinuxMountContext="" Oct 30 22:08:32.126584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126620 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="df6b6cae-173c-461c-9195-4dce90bb6656" volumeName="kubernetes.io/secret/df6b6cae-173c-461c-9195-4dce90bb6656-kube-state-metrics-tls" seLinuxMountContext="" Oct 30 22:08:32.128828 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126640 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3155dfa9-0dc2-4256-9e16-490b3721702d" volumeName="kubernetes.io/projected/3155dfa9-0dc2-4256-9e16-490b3721702d-kube-api-access-2gvkg" seLinuxMountContext="" Oct 30 22:08:32.128828 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126924 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" volumeName="kubernetes.io/projected/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-kube-api-access-bpcht" seLinuxMountContext="" Oct 30 22:08:32.128828 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126943 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="11464051-0aa4-4324-b2c7-7ea2096bbed6" volumeName="kubernetes.io/projected/11464051-0aa4-4324-b2c7-7ea2096bbed6-kube-api-access-nrwhs" seLinuxMountContext="" Oct 30 22:08:32.128828 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126968 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3913a2c9-e88c-4f49-bc63-02b1fa543c38" volumeName="kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-thanos-prometheus-http-client-file" seLinuxMountContext="" Oct 30 22:08:32.128828 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126981 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="68935e75-a4f0-4a54-9094-4f12b4a2dc39" volumeName="kubernetes.io/projected/68935e75-a4f0-4a54-9094-4f12b4a2dc39-kube-api-access-z6fv5" seLinuxMountContext="" Oct 30 22:08:32.128828 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.126998 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="94283f0e-a719-4152-aea8-e219f7bb7eef" volumeName="kubernetes.io/projected/94283f0e-a719-4152-aea8-e219f7bb7eef-kube-api-access-k2kkt" seLinuxMountContext="" Oct 30 22:08:32.128828 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.127013 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1b444016-092c-472b-979a-4a7b16271e27" volumeName="kubernetes.io/secret/1b444016-092c-472b-979a-4a7b16271e27-secret-thanos-querier-kube-rbac-proxy-rules" seLinuxMountContext="" Oct 30 22:08:32.128828 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.127027 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="68935e75-a4f0-4a54-9094-4f12b4a2dc39" volumeName="kubernetes.io/secret/68935e75-a4f0-4a54-9094-4f12b4a2dc39-node-exporter-tls" seLinuxMountContext="" Oct 30 22:08:32.128828 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.127045 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f6cc017b-3f50-423a-93da-e4515336a2bc" volumeName="kubernetes.io/secret/f6cc017b-3f50-423a-93da-e4515336a2bc-agent-certs" seLinuxMountContext="" Oct 30 22:08:32.128828 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.127057 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" volumeName="kubernetes.io/empty-dir/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-alertmanager-main-db" seLinuxMountContext="" Oct 30 22:08:32.128828 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.127075 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3913a2c9-e88c-4f49-bc63-02b1fa543c38" volumeName="kubernetes.io/projected/3913a2c9-e88c-4f49-bc63-02b1fa543c38-tls-assets" seLinuxMountContext="" Oct 30 22:08:32.128828 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.127087 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" volumeName="kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-secret-alertmanager-kube-rbac-proxy" seLinuxMountContext="" Oct 30 22:08:32.128828 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.127100 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c373c767-d743-4819-be7c-4c9c830eb9db" volumeName="kubernetes.io/projected/c373c767-d743-4819-be7c-4c9c830eb9db-kube-api-access-cj545" seLinuxMountContext="" Oct 30 22:08:32.128828 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.127122 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e129700d-0583-4316-bcee-431c57b0effa" volumeName="kubernetes.io/configmap/e129700d-0583-4316-bcee-431c57b0effa-metrics-client-ca" seLinuxMountContext="" Oct 30 22:08:32.128828 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.127138 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44" volumeName="kubernetes.io/empty-dir/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-etc-tuned" seLinuxMountContext="" Oct 30 22:08:32.128828 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.127174 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3913a2c9-e88c-4f49-bc63-02b1fa543c38" volumeName="kubernetes.io/empty-dir/3913a2c9-e88c-4f49-bc63-02b1fa543c38-config-out" seLinuxMountContext="" Oct 30 22:08:32.128828 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.127185 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1b444016-092c-472b-979a-4a7b16271e27" volumeName="kubernetes.io/secret/1b444016-092c-472b-979a-4a7b16271e27-secret-thanos-querier-kube-rbac-proxy-metrics" seLinuxMountContext="" Oct 30 22:08:32.128828 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.127200 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="68935e75-a4f0-4a54-9094-4f12b4a2dc39" volumeName="kubernetes.io/configmap/68935e75-a4f0-4a54-9094-4f12b4a2dc39-metrics-client-ca" seLinuxMountContext="" Oct 30 22:08:32.128828 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.127240 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3913a2c9-e88c-4f49-bc63-02b1fa543c38" volumeName="kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-configmap-metrics-client-ca" seLinuxMountContext="" Oct 30 22:08:32.128828 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.127255 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" volumeName="kubernetes.io/configmap/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-alertmanager-trusted-ca-bundle" seLinuxMountContext="" Oct 30 22:08:32.128828 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.127272 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3913a2c9-e88c-4f49-bc63-02b1fa543c38" volumeName="kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-prometheus-k8s-kube-rbac-proxy-web" seLinuxMountContext="" Oct 30 22:08:32.128828 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.127286 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" volumeName="kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-config-volume" seLinuxMountContext="" Oct 30 22:08:32.128828 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.127298 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="68935e75-a4f0-4a54-9094-4f12b4a2dc39" volumeName="kubernetes.io/empty-dir/68935e75-a4f0-4a54-9094-4f12b4a2dc39-node-exporter-textfile" seLinuxMountContext="" Oct 30 22:08:32.128828 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.127316 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="94283f0e-a719-4152-aea8-e219f7bb7eef" volumeName="kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-oauth-config" seLinuxMountContext="" Oct 30 22:08:32.128828 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.127328 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c9381d26-e2af-4730-8b4e-06945c59585c" volumeName="kubernetes.io/configmap/c9381d26-e2af-4730-8b4e-06945c59585c-whereabouts-flatfile-configmap" seLinuxMountContext="" Oct 30 22:08:32.128828 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.127392 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="11464051-0aa4-4324-b2c7-7ea2096bbed6" volumeName="kubernetes.io/configmap/11464051-0aa4-4324-b2c7-7ea2096bbed6-serviceca" seLinuxMountContext="" Oct 30 22:08:32.128828 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.127413 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3913a2c9-e88c-4f49-bc63-02b1fa543c38" volumeName="kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-trusted-ca-bundle" seLinuxMountContext="" Oct 30 22:08:32.128828 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.127427 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="94283f0e-a719-4152-aea8-e219f7bb7eef" volumeName="kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-service-ca" seLinuxMountContext="" Oct 30 22:08:32.128828 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.127450 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c9381d26-e2af-4730-8b4e-06945c59585c" volumeName="kubernetes.io/configmap/c9381d26-e2af-4730-8b4e-06945c59585c-cni-sysctl-allowlist" seLinuxMountContext="" Oct 30 22:08:32.128828 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.127468 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" volumeName="kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-web-config" seLinuxMountContext="" Oct 30 22:08:32.128828 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.127483 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1b444016-092c-472b-979a-4a7b16271e27" volumeName="kubernetes.io/secret/1b444016-092c-472b-979a-4a7b16271e27-secret-thanos-querier-kube-rbac-proxy-web" seLinuxMountContext="" Oct 30 22:08:32.128828 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.127504 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f64980b7-13c0-40c3-aa3f-277916f24518" volumeName="kubernetes.io/projected/f64980b7-13c0-40c3-aa3f-277916f24518-kube-api-access-tjk5b" seLinuxMountContext="" Oct 30 22:08:32.128828 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.127517 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="94283f0e-a719-4152-aea8-e219f7bb7eef" volumeName="kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-trusted-ca-bundle" seLinuxMountContext="" Oct 30 22:08:32.128828 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.127536 5949 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c373c767-d743-4819-be7c-4c9c830eb9db" volumeName="kubernetes.io/configmap/c373c767-d743-4819-be7c-4c9c830eb9db-config-volume" seLinuxMountContext="" Oct 30 22:08:32.128828 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.127548 5949 reconstruct.go:97] "Volume reconstruction finished" Oct 30 22:08:32.128828 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.127555 5949 reconciler.go:29] "Reconciler: start to sync state" Oct 30 22:08:32.151283 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.151181 5949 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Oct 30 22:08:32.154279 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.154257 5949 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Oct 30 22:08:32.154279 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.154277 5949 status_manager.go:249] "Starting to sync pod status with apiserver" Oct 30 22:08:32.154279 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.154294 5949 kubelet.go:2521] "Starting kubelet main sync loop" Oct 30 22:08:32.154508 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:32.154326 5949 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Oct 30 22:08:32.160307 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.160283 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ql6lh_074eeae7-932c-4642-8fea-671898135ab1/ovn-acl-logging/0.log" Oct 30 22:08:32.164110 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.164080 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ql6lh_074eeae7-932c-4642-8fea-671898135ab1/ovn-acl-logging/0.log" Oct 30 22:08:32.164932 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.164859 5949 generic.go:356] "Generic (PLEG): container finished" podID="3913a2c9-e88c-4f49-bc63-02b1fa543c38" containerID="8b9b1cb2d4b58b1640e7b7ffa96201c957c0f209ed4cbb0101202683c3db5f62" exitCode=0 Oct 30 22:08:32.165882 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.165398 5949 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 22:08:32.165882 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.165480 5949 reconstruct.go:205] "DevicePaths of reconstructed volumes updated" Oct 30 22:08:32.166578 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.166561 5949 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 22:08:32.172587 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.172556 5949 generic.go:356] "Generic (PLEG): container finished" podID="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" containerID="e599f9a04f1e39f29f36cd26b16b488cf06bbe03c9228c4fda6d8f10e43c0b31" exitCode=0 Oct 30 22:08:32.186895 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.186873 5949 generic.go:356] "Generic (PLEG): container finished" podID="c9381d26-e2af-4730-8b4e-06945c59585c" containerID="e95cee11b495d8aa21d824e37929e77e6d965e46f61d7d2bbb52b7b5d92e031b" exitCode=0 Oct 30 22:08:32.186895 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.186892 5949 generic.go:356] "Generic (PLEG): container finished" podID="c9381d26-e2af-4730-8b4e-06945c59585c" containerID="10a0c9b2bbd7ee49f4ebc0f3e62b685d09bd313f36fc21b1de82b366a99b2917" exitCode=0 Oct 30 22:08:32.187059 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.186905 5949 generic.go:356] "Generic (PLEG): container finished" podID="c9381d26-e2af-4730-8b4e-06945c59585c" containerID="7c3a93c19db6632b6bb5575a3480ad2072ed954a14068dffd75a26f4c51ed077" exitCode=0 Oct 30 22:08:32.187059 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.186914 5949 generic.go:356] "Generic (PLEG): container finished" podID="c9381d26-e2af-4730-8b4e-06945c59585c" containerID="f5892fef9173fe1fab830db1d7a0a6deb7dd59be988e2390f945b5c3da199d30" exitCode=0 Oct 30 22:08:32.187059 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.186923 5949 generic.go:356] "Generic (PLEG): container finished" podID="c9381d26-e2af-4730-8b4e-06945c59585c" containerID="373a51f4307dda59159c6142c60b1cb226d82e6d46f1df486ade756a7db85d5d" exitCode=0 Oct 30 22:08:32.187059 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.186932 5949 generic.go:356] "Generic (PLEG): container finished" podID="c9381d26-e2af-4730-8b4e-06945c59585c" containerID="3528fd4933a28b3cad22eae8fc5e5d8ce04a20a2ac0830f643908749a15990b8" exitCode=0 Oct 30 22:08:32.188717 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.188698 5949 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 22:08:32.188814 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.188703 5949 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 22:08:32.205513 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.205487 5949 generic.go:356] "Generic (PLEG): container finished" podID="68935e75-a4f0-4a54-9094-4f12b4a2dc39" containerID="6814454a571673b27d352100071f0cc3eb1aa6bad6f3f89ace251007ca7e8b5c" exitCode=0 Oct 30 22:08:32.214016 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.213999 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ql6lh_074eeae7-932c-4642-8fea-671898135ab1/ovn-acl-logging/0.log" Oct 30 22:08:32.214356 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.214334 5949 generic.go:356] "Generic (PLEG): container finished" podID="074eeae7-932c-4642-8fea-671898135ab1" containerID="faf0c33ee470fd578edf1aeac1625ea138771e550ea0c26c6ba30af959ef8352" exitCode=1 Oct 30 22:08:32.217432 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.217409 5949 generic.go:356] "Generic (PLEG): container finished" podID="203b93f5f211a36c6aba561c49a0c709" containerID="d8b95a78a3a21da3a8b173742807aea40466b40099b2dff7e76c6ef962e1d437" exitCode=0 Oct 30 22:08:32.248094 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.248031 5949 manager.go:324] Recovery completed Oct 30 22:08:32.254423 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:32.254400 5949 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet" Oct 30 22:08:32.278073 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.278050 5949 cpu_manager.go:226] "Starting" policy="none" Oct 30 22:08:32.278073 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.278067 5949 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Oct 30 22:08:32.278198 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.278089 5949 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Oct 30 22:08:32.278249 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.278210 5949 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet="" Oct 30 22:08:32.278249 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.278234 5949 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={} Oct 30 22:08:32.278305 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.278251 5949 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint" Oct 30 22:08:32.278305 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.278257 5949 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet="" Oct 30 22:08:32.278305 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.278262 5949 policy_none.go:50] "Start" Oct 30 22:08:32.278305 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.278271 5949 memory_manager.go:187] "Starting memorymanager" policy="None" Oct 30 22:08:32.278305 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.278279 5949 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Oct 30 22:08:32.278436 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.278399 5949 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint" Oct 30 22:08:32.278436 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.278406 5949 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint" Oct 30 22:08:32.278436 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.278411 5949 policy_none.go:44] "Start" Oct 30 22:08:32.283262 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.283230 5949 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Oct 30 22:08:32.287063 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.287045 5949 manager.go:341] "Starting Device Plugin manager" Oct 30 22:08:32.287778 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:32.287076 5949 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Oct 30 22:08:32.287831 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.287782 5949 server.go:87] "Starting device plugin registration server" Oct 30 22:08:32.288084 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.288072 5949 eviction_manager.go:194] "Eviction manager: starting control loop" Oct 30 22:08:32.288124 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.288086 5949 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Oct 30 22:08:32.288191 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.288174 5949 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Oct 30 22:08:32.288365 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.288314 5949 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Oct 30 22:08:32.288468 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.288369 5949 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Oct 30 22:08:32.288468 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.288376 5949 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Oct 30 22:08:32.288553 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.288455 5949 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T22:08:32.288359759Z","UUID":"c31fda53-4074-46d1-9b1a-4739f2e188e4","Handler":null,"Name":"","Endpoint":""} Oct 30 22:08:32.290266 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:32.290244 5949 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Oct 30 22:08:32.293138 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.293106 5949 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Oct 30 22:08:32.293262 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.293159 5949 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Oct 30 22:08:32.388534 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.388498 5949 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 22:08:32.390459 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.390439 5949 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-24.ec2.internal" event="NodeHasSufficientMemory" Oct 30 22:08:32.390570 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.390473 5949 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-24.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 22:08:32.390570 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.390488 5949 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-24.ec2.internal" event="NodeHasSufficientPID" Oct 30 22:08:32.390570 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.390548 5949 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-142-24.ec2.internal" Oct 30 22:08:32.423669 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.423649 5949 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-142-24.ec2.internal" Oct 30 22:08:32.423760 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.423706 5949 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-142-24.ec2.internal" Oct 30 22:08:32.455197 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.455152 5949 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-142-24.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal"] Oct 30 22:08:32.455793 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.455776 5949 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-24.ec2.internal" Oct 30 22:08:32.455957 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.455905 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-24.ec2.internal" event={"ID":"7655e583c88021e605aee040501f02ec","Type":"ContainerStarted","Data":"f583ef41b14444a12619d158a85c6ae15003034ee7466bd2ddbc5883873a5b45"} Oct 30 22:08:32.456001 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.455974 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-24.ec2.internal" event={"ID":"7655e583c88021e605aee040501f02ec","Type":"ContainerStarted","Data":"36051ba353f4d1493e47afd3bee1fa39336698eac55c1ddd7a97c40dbd52bdfa"} Oct 30 22:08:32.456066 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.456007 5949 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal" Oct 30 22:08:32.456101 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.456006 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal" event={"ID":"203b93f5f211a36c6aba561c49a0c709","Type":"ContainerStarted","Data":"c7f1875734cc72055502d36d4ad093f78d6b2d6782061c779b58ca176e028ce0"} Oct 30 22:08:32.456135 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.456113 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal" event={"ID":"203b93f5f211a36c6aba561c49a0c709","Type":"ContainerDied","Data":"d8b95a78a3a21da3a8b173742807aea40466b40099b2dff7e76c6ef962e1d437"} Oct 30 22:08:32.456164 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.456135 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal" event={"ID":"203b93f5f211a36c6aba561c49a0c709","Type":"ContainerStarted","Data":"61a90835359d781127b7c1b0c8148cf6aef8e3c7ad74148743763ac42eb9b7ad"} Oct 30 22:08:32.480975 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.480945 5949 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 22:08:32.481071 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:32.480997 5949 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-142-24.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-24.ec2.internal" Oct 30 22:08:32.481671 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.481656 5949 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 22:08:32.481706 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:32.481693 5949 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal" Oct 30 22:08:32.568082 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.567974 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7655e583c88021e605aee040501f02ec-config\") pod \"kube-apiserver-proxy-ip-10-0-142-24.ec2.internal\" (UID: \"7655e583c88021e605aee040501f02ec\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-24.ec2.internal" Oct 30 22:08:32.568082 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.568053 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/203b93f5f211a36c6aba561c49a0c709-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal\" (UID: \"203b93f5f211a36c6aba561c49a0c709\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal" Oct 30 22:08:32.568211 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.568087 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/203b93f5f211a36c6aba561c49a0c709-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal\" (UID: \"203b93f5f211a36c6aba561c49a0c709\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal" Oct 30 22:08:32.668582 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.668558 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7655e583c88021e605aee040501f02ec-config\") pod \"kube-apiserver-proxy-ip-10-0-142-24.ec2.internal\" (UID: \"7655e583c88021e605aee040501f02ec\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-24.ec2.internal" Oct 30 22:08:32.668689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.668587 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/203b93f5f211a36c6aba561c49a0c709-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal\" (UID: \"203b93f5f211a36c6aba561c49a0c709\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal" Oct 30 22:08:32.668689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.668615 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/203b93f5f211a36c6aba561c49a0c709-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal\" (UID: \"203b93f5f211a36c6aba561c49a0c709\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal" Oct 30 22:08:32.668689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.668662 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/203b93f5f211a36c6aba561c49a0c709-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal\" (UID: \"203b93f5f211a36c6aba561c49a0c709\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal" Oct 30 22:08:32.668689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.668667 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7655e583c88021e605aee040501f02ec-config\") pod \"kube-apiserver-proxy-ip-10-0-142-24.ec2.internal\" (UID: \"7655e583c88021e605aee040501f02ec\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-24.ec2.internal" Oct 30 22:08:32.668816 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:32.668705 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/203b93f5f211a36c6aba561c49a0c709-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal\" (UID: \"203b93f5f211a36c6aba561c49a0c709\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal" Oct 30 22:08:33.104385 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.104356 5949 apiserver.go:52] "Watching apiserver" Oct 30 22:08:33.154786 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.154756 5949 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Oct 30 22:08:33.155424 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.155401 5949 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-142-24.ec2.internal","openshift-network-diagnostics/network-check-target-knzz5","openshift-dns/dns-default-pl4x2","openshift-dns/node-resolver-pnr6k","openshift-image-registry/node-ca-dbtlp","openshift-ovn-kubernetes/ovnkube-node-ql6lh","openshift-cluster-node-tuning-operator/tuned-8hv7m","openshift-insights/insights-runtime-extractor-pn4kg","openshift-monitoring/kube-state-metrics-5dc8d7d89b-6krqs","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc","openshift-monitoring/metrics-server-f89f5d58d-95dk2","openshift-multus/network-metrics-daemon-l7kkv","openshift-network-operator/iptables-alerter-b68bn","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal","openshift-monitoring/node-exporter-s7svn","openshift-monitoring/openshift-state-metrics-7bd7489957-xs8x9","openshift-multus/multus-f6888","openshift-console/console-6d4659d6dd-rb87s","openshift-monitoring/monitoring-plugin-77f4f76594-29lx9","openshift-monitoring/prometheus-k8s-0","openshift-monitoring/thanos-querier-5ddf98bd44-xfj64","openshift-ingress-canary/ingress-canary-9mkk7","openshift-monitoring/alertmanager-main-0","kube-system/global-pull-secret-syncer-h9v2l","openshift-monitoring/telemeter-client-74754b6cd9-49r6n","openshift-multus/multus-additional-cni-plugins-v75vr","kube-system/konnectivity-agent-zlbzf"] Oct 30 22:08:33.171125 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.171100 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Oct 30 22:08:33.171125 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.171110 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Oct 30 22:08:33.171301 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.171161 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Oct 30 22:08:33.171938 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.171918 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Oct 30 22:08:33.172852 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.172828 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Oct 30 22:08:33.172954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.172859 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Oct 30 22:08:33.173147 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.173128 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-s9t4n\"" Oct 30 22:08:33.173200 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.173151 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Oct 30 22:08:33.173317 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.173300 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Oct 30 22:08:33.173490 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.173468 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-22mqf\"" Oct 30 22:08:33.173598 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.173490 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Oct 30 22:08:33.173598 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.173510 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Oct 30 22:08:33.173755 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.173737 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-x4nws\"" Oct 30 22:08:33.173878 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.173857 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Oct 30 22:08:33.173927 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.173917 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-pjfxj\"" Oct 30 22:08:33.173996 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.173980 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Oct 30 22:08:33.174045 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.174009 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Oct 30 22:08:33.174045 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.174030 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Oct 30 22:08:33.174127 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.174106 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Oct 30 22:08:33.174234 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.174200 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-hmbsc\"" Oct 30 22:08:33.174332 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.174313 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Oct 30 22:08:33.182958 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.182940 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Oct 30 22:08:33.183085 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.182970 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Oct 30 22:08:33.183085 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.183074 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Oct 30 22:08:33.183085 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.183035 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Oct 30 22:08:33.183305 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.182998 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Oct 30 22:08:33.183305 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.183159 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Oct 30 22:08:33.183305 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.183285 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Oct 30 22:08:33.183305 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.183296 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Oct 30 22:08:33.183471 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.183296 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Oct 30 22:08:33.183471 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.183317 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Oct 30 22:08:33.183471 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.183290 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Oct 30 22:08:33.183471 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.183387 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zgfdn\"" Oct 30 22:08:33.183471 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.183435 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Oct 30 22:08:33.183471 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.183440 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Oct 30 22:08:33.183680 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.183498 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Oct 30 22:08:33.183680 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.183499 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gfzqs\"" Oct 30 22:08:33.183742 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.183681 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Oct 30 22:08:33.183792 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.183753 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Oct 30 22:08:33.183841 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.183799 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Oct 30 22:08:33.183889 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.183859 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Oct 30 22:08:33.183889 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.183876 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Oct 30 22:08:33.183969 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.183913 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Oct 30 22:08:33.184012 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.183966 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Oct 30 22:08:33.184012 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.183975 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Oct 30 22:08:33.184012 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.184005 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-t24hw\"" Oct 30 22:08:33.184144 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.184079 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Oct 30 22:08:33.184144 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.184091 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-5s2rl\"" Oct 30 22:08:33.184144 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.184102 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Oct 30 22:08:33.184438 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.184382 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Oct 30 22:08:33.184438 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.184388 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Oct 30 22:08:33.184438 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.184411 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Oct 30 22:08:33.184438 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.184418 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5nx2r\"" Oct 30 22:08:33.184438 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.184416 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Oct 30 22:08:33.184438 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.184427 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Oct 30 22:08:33.184438 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.184473 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Oct 30 22:08:33.184438 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.184481 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hk5lx\"" Oct 30 22:08:33.184438 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.184535 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Oct 30 22:08:33.185399 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.184579 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Oct 30 22:08:33.185399 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.184644 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Oct 30 22:08:33.185399 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.184669 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Oct 30 22:08:33.185399 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.184684 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Oct 30 22:08:33.185399 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.185054 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Oct 30 22:08:33.185927 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.185914 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Oct 30 22:08:33.185988 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.185962 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-vdq5j\"" Oct 30 22:08:33.186039 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.186009 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Oct 30 22:08:33.186085 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.186064 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Oct 30 22:08:33.186158 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.186143 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-b6lz9\"" Oct 30 22:08:33.186207 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.186195 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-r7nlg\"" Oct 30 22:08:33.186389 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.186361 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Oct 30 22:08:33.186389 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.186371 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Oct 30 22:08:33.186389 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.186363 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Oct 30 22:08:33.194707 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.194685 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-jt6r2\"" Oct 30 22:08:33.194909 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.194694 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-buf3qpeaoqr6i\"" Oct 30 22:08:33.195807 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.195787 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Oct 30 22:08:33.195956 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.195814 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Oct 30 22:08:33.195956 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.195827 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Oct 30 22:08:33.196068 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.195988 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Oct 30 22:08:33.196068 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.196000 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-dzfp9\"" Oct 30 22:08:33.196383 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.196369 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Oct 30 22:08:33.196424 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.196379 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-gz6qb\"" Oct 30 22:08:33.196653 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.196635 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Oct 30 22:08:33.196653 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.196640 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Oct 30 22:08:33.196765 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.196671 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Oct 30 22:08:33.196765 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.196676 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Oct 30 22:08:33.196840 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.196827 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Oct 30 22:08:33.196929 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.196910 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Oct 30 22:08:33.196987 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.196929 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Oct 30 22:08:33.196987 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.196928 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Oct 30 22:08:33.196987 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.196947 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Oct 30 22:08:33.196987 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.196928 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-r6g4b\"" Oct 30 22:08:33.196987 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.196971 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-7vf6q\"" Oct 30 22:08:33.196987 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.196954 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Oct 30 22:08:33.196987 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.196966 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Oct 30 22:08:33.197205 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.196992 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Oct 30 22:08:33.197205 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.197038 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Oct 30 22:08:33.197205 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.197060 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-p4pd4\"" Oct 30 22:08:33.197205 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.197182 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-vvmekr9k6f8j\"" Oct 30 22:08:33.197387 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.197278 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Oct 30 22:08:33.197387 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.197313 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Oct 30 22:08:33.197471 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.197456 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\"" Oct 30 22:08:33.197704 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.197688 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Oct 30 22:08:33.197852 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.197837 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-7l7rd\"" Oct 30 22:08:33.197900 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.197889 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Oct 30 22:08:33.197962 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.197951 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-gq78g\"" Oct 30 22:08:33.198290 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.198276 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Oct 30 22:08:33.198441 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.198427 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Oct 30 22:08:33.224806 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.224770 5949 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Oct 30 22:08:33.229557 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.229536 5949 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-24.ec2.internal" Oct 30 22:08:33.229684 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.229659 5949 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal" Oct 30 22:08:33.252068 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.252044 5949 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 22:08:33.252494 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:33.252470 5949 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-142-24.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-24.ec2.internal" Oct 30 22:08:33.256236 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.256201 5949 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 22:08:33.256337 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:33.256270 5949 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal" Oct 30 22:08:33.270632 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.270610 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb3d3e48-51f2-4503-8026-79787ee98754-telemeter-trusted-ca-bundle\") pod \"telemeter-client-74754b6cd9-49r6n\" (UID: \"fb3d3e48-51f2-4503-8026-79787ee98754\") " pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" Oct 30 22:08:33.270680 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.270637 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:33.270680 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.270654 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:33.270763 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.270675 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-host-var-lib-kubelet\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.270763 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.270704 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/fb3d3e48-51f2-4503-8026-79787ee98754-federate-client-tls\") pod \"telemeter-client-74754b6cd9-49r6n\" (UID: \"fb3d3e48-51f2-4503-8026-79787ee98754\") " pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" Oct 30 22:08:33.270763 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.270728 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/074eeae7-932c-4642-8fea-671898135ab1-env-overrides\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.270763 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.270754 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/df6b6cae-173c-461c-9195-4dce90bb6656-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-6krqs\" (UID: \"df6b6cae-173c-461c-9195-4dce90bb6656\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6krqs" Oct 30 22:08:33.270949 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.270787 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11464051-0aa4-4324-b2c7-7ea2096bbed6-host\") pod \"node-ca-dbtlp\" (UID: \"11464051-0aa4-4324-b2c7-7ea2096bbed6\") " pod="openshift-image-registry/node-ca-dbtlp" Oct 30 22:08:33.270949 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.270811 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:33.270949 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.270931 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cd46f3e8-c752-4eed-a65d-07e060bf32b4-kubelet-config\") pod \"global-pull-secret-syncer-h9v2l\" (UID: \"cd46f3e8-c752-4eed-a65d-07e060bf32b4\") " pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:08:33.271109 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.271030 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:33.271109 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.271074 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb3d3e48-51f2-4503-8026-79787ee98754-telemeter-trusted-ca-bundle\") pod \"telemeter-client-74754b6cd9-49r6n\" (UID: \"fb3d3e48-51f2-4503-8026-79787ee98754\") " pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" Oct 30 22:08:33.271183 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.271095 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/074eeae7-932c-4642-8fea-671898135ab1-env-overrides\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.271183 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.271122 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/fb3d3e48-51f2-4503-8026-79787ee98754-federate-client-tls\") pod \"telemeter-client-74754b6cd9-49r6n\" (UID: \"fb3d3e48-51f2-4503-8026-79787ee98754\") " pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" Oct 30 22:08:33.271303 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.271288 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/df6b6cae-173c-461c-9195-4dce90bb6656-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-6krqs\" (UID: \"df6b6cae-173c-461c-9195-4dce90bb6656\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6krqs" Oct 30 22:08:33.271365 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.271307 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Oct 30 22:08:33.271365 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.271310 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Oct 30 22:08:33.271365 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.271358 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Oct 30 22:08:33.271502 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.271362 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b444016-092c-472b-979a-4a7b16271e27-metrics-client-ca\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:33.271502 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.271435 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68935e75-a4f0-4a54-9094-4f12b4a2dc39-metrics-client-ca\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:33.271502 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.271446 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Oct 30 22:08:33.271620 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.271503 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5bb239d5-4f0e-4dd6-b58b-5f3b9e014895-iptables-alerter-script\") pod \"iptables-alerter-b68bn\" (UID: \"5bb239d5-4f0e-4dd6-b58b-5f3b9e014895\") " pod="openshift-network-operator/iptables-alerter-b68bn" Oct 30 22:08:33.271620 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.271506 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-anvv041vkp1fj\"" Oct 30 22:08:33.271620 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.271577 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Oct 30 22:08:33.271620 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.271591 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b444016-092c-472b-979a-4a7b16271e27-metrics-client-ca\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:33.271620 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.271609 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Oct 30 22:08:33.271850 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.271673 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-jctzf\"" Oct 30 22:08:33.271850 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.271676 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-log-socket\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.271850 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.271701 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5bb239d5-4f0e-4dd6-b58b-5f3b9e014895-iptables-alerter-script\") pod \"iptables-alerter-b68bn\" (UID: \"5bb239d5-4f0e-4dd6-b58b-5f3b9e014895\") " pod="openshift-network-operator/iptables-alerter-b68bn" Oct 30 22:08:33.271850 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.271701 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68935e75-a4f0-4a54-9094-4f12b4a2dc39-metrics-client-ca\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:33.271850 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.271732 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/68935e75-a4f0-4a54-9094-4f12b4a2dc39-root\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:33.271850 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.271775 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-pgqsk\"" Oct 30 22:08:33.271850 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.271782 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6fv5\" (UniqueName: \"kubernetes.io/projected/68935e75-a4f0-4a54-9094-4f12b4a2dc39-kube-api-access-z6fv5\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:33.271850 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.271828 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Oct 30 22:08:33.272116 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.271884 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9862\" (UniqueName: \"kubernetes.io/projected/1b444016-092c-472b-979a-4a7b16271e27-kube-api-access-j9862\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:33.272116 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.271957 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-multus-cni-dir\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.272116 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.272011 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3913a2c9-e88c-4f49-bc63-02b1fa543c38-config-out\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:33.272116 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.272072 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5dfaf077-3637-4cc9-8681-466c4e2cc5f4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pn4kg\" (UID: \"5dfaf077-3637-4cc9-8681-466c4e2cc5f4\") " pod="openshift-insights/insights-runtime-extractor-pn4kg" Oct 30 22:08:33.272116 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.272099 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3913a2c9-e88c-4f49-bc63-02b1fa543c38-config-out\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:33.272281 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.272153 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gvkg\" (UniqueName: \"kubernetes.io/projected/3155dfa9-0dc2-4256-9e16-490b3721702d-kube-api-access-2gvkg\") pod \"network-check-target-knzz5\" (UID: \"3155dfa9-0dc2-4256-9e16-490b3721702d\") " pod="openshift-network-diagnostics/network-check-target-knzz5" Oct 30 22:08:33.272281 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.272252 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-system-cni-dir\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.272348 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.272323 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5dfaf077-3637-4cc9-8681-466c4e2cc5f4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pn4kg\" (UID: \"5dfaf077-3637-4cc9-8681-466c4e2cc5f4\") " pod="openshift-insights/insights-runtime-extractor-pn4kg" Oct 30 22:08:33.272391 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.272370 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b042dce-1fff-41a2-83dd-ae18f3098207-tmp-dir\") pod \"node-resolver-pnr6k\" (UID: \"6b042dce-1fff-41a2-83dd-ae18f3098207\") " pod="openshift-dns/node-resolver-pnr6k" Oct 30 22:08:33.272421 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.272409 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-config-out\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:33.272453 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.272436 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:33.272486 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.272461 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cd46f3e8-c752-4eed-a65d-07e060bf32b4-dbus\") pod \"global-pull-secret-syncer-h9v2l\" (UID: \"cd46f3e8-c752-4eed-a65d-07e060bf32b4\") " pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:08:33.272486 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.272466 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b042dce-1fff-41a2-83dd-ae18f3098207-tmp-dir\") pod \"node-resolver-pnr6k\" (UID: \"6b042dce-1fff-41a2-83dd-ae18f3098207\") " pod="openshift-dns/node-resolver-pnr6k" Oct 30 22:08:33.272550 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.272489 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/1b444016-092c-472b-979a-4a7b16271e27-secret-thanos-querier-tls\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:33.272550 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.272517 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1b444016-092c-472b-979a-4a7b16271e27-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:33.272659 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.272549 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-config-out\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:33.272659 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.272551 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/1b444016-092c-472b-979a-4a7b16271e27-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:33.272659 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.272595 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxdbc\" (UniqueName: \"kubernetes.io/projected/6b042dce-1fff-41a2-83dd-ae18f3098207-kube-api-access-qxdbc\") pod \"node-resolver-pnr6k\" (UID: \"6b042dce-1fff-41a2-83dd-ae18f3098207\") " pod="openshift-dns/node-resolver-pnr6k" Oct 30 22:08:33.272659 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.272631 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/68935e75-a4f0-4a54-9094-4f12b4a2dc39-node-exporter-wtmp\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:33.272850 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.272662 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:33.272850 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.272671 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:33.272850 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.272690 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c9381d26-e2af-4730-8b4e-06945c59585c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v75vr\" (UID: \"c9381d26-e2af-4730-8b4e-06945c59585c\") " pod="openshift-multus/multus-additional-cni-plugins-v75vr" Oct 30 22:08:33.272850 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.272708 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/1b444016-092c-472b-979a-4a7b16271e27-secret-thanos-querier-tls\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:33.272850 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.272728 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9ac70a73-f7f0-4b3e-bf76-09869caefea0-client-ca-bundle\") pod \"metrics-server-f89f5d58d-95dk2\" (UID: \"9ac70a73-f7f0-4b3e-bf76-09869caefea0\") " pod="openshift-monitoring/metrics-server-f89f5d58d-95dk2" Oct 30 22:08:33.272850 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.272755 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/1b444016-092c-472b-979a-4a7b16271e27-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:33.272850 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.272757 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-host-var-lib-cni-multus\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.272850 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.272768 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1b444016-092c-472b-979a-4a7b16271e27-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:33.272850 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.272816 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f6cc017b-3f50-423a-93da-e4515336a2bc-agent-certs\") pod \"konnectivity-agent-zlbzf\" (UID: \"f6cc017b-3f50-423a-93da-e4515336a2bc\") " pod="kube-system/konnectivity-agent-zlbzf" Oct 30 22:08:33.272850 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.272845 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/68935e75-a4f0-4a54-9094-4f12b4a2dc39-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:33.273346 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.272861 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c9381d26-e2af-4730-8b4e-06945c59585c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v75vr\" (UID: \"c9381d26-e2af-4730-8b4e-06945c59585c\") " pod="openshift-multus/multus-additional-cni-plugins-v75vr" Oct 30 22:08:33.273346 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.272871 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-trusted-ca-bundle\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:33.273346 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.272960 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9ac70a73-f7f0-4b3e-bf76-09869caefea0-client-ca-bundle\") pod \"metrics-server-f89f5d58d-95dk2\" (UID: \"9ac70a73-f7f0-4b3e-bf76-09869caefea0\") " pod="openshift-monitoring/metrics-server-f89f5d58d-95dk2" Oct 30 22:08:33.273346 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.272994 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7f59\" (UniqueName: \"kubernetes.io/projected/e5779bc8-21ad-465e-8749-8f7e9afdc03f-kube-api-access-h7f59\") pod \"aws-ebs-csi-driver-node-c69jc\" (UID: \"e5779bc8-21ad-465e-8749-8f7e9afdc03f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" Oct 30 22:08:33.273346 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:33.273006 5949 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered Oct 30 22:08:33.273346 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.273024 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-multus-conf-dir\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.273346 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.273052 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb3d3e48-51f2-4503-8026-79787ee98754-metrics-client-ca\") pod \"telemeter-client-74754b6cd9-49r6n\" (UID: \"fb3d3e48-51f2-4503-8026-79787ee98754\") " pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" Oct 30 22:08:33.273346 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.273072 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f6cc017b-3f50-423a-93da-e4515336a2bc-agent-certs\") pod \"konnectivity-agent-zlbzf\" (UID: \"f6cc017b-3f50-423a-93da-e4515336a2bc\") " pod="kube-system/konnectivity-agent-zlbzf" Oct 30 22:08:33.273346 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.273074 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:33.273346 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:33.273077 5949 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-trusted-ca-bundle podName:94283f0e-a719-4152-aea8-e219f7bb7eef nodeName:}" failed. No retries permitted until 2026-10-30 22:08:33.773056186 +0000 UTC m=+1.736864751 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-trusted-ca-bundle") pod "console-6d4659d6dd-rb87s" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef") : object "openshift-console"/"trusted-ca-bundle" not registered Oct 30 22:08:33.273346 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.273113 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c373c767-d743-4819-be7c-4c9c830eb9db-metrics-tls\") pod \"dns-default-pl4x2\" (UID: \"c373c767-d743-4819-be7c-4c9c830eb9db\") " pod="openshift-dns/dns-default-pl4x2" Oct 30 22:08:33.273346 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.273133 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:33.273346 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.273147 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/11464051-0aa4-4324-b2c7-7ea2096bbed6-serviceca\") pod \"node-ca-dbtlp\" (UID: \"11464051-0aa4-4324-b2c7-7ea2096bbed6\") " pod="openshift-image-registry/node-ca-dbtlp" Oct 30 22:08:33.273346 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.273166 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfsr2\" (UniqueName: \"kubernetes.io/projected/9ac70a73-f7f0-4b3e-bf76-09869caefea0-kube-api-access-bfsr2\") pod \"metrics-server-f89f5d58d-95dk2\" (UID: \"9ac70a73-f7f0-4b3e-bf76-09869caefea0\") " pod="openshift-monitoring/metrics-server-f89f5d58d-95dk2" Oct 30 22:08:33.273346 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.273141 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/68935e75-a4f0-4a54-9094-4f12b4a2dc39-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:33.273346 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.273185 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:33.273346 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.273242 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-multus-daemon-config\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.273346 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.273253 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb3d3e48-51f2-4503-8026-79787ee98754-metrics-client-ca\") pod \"telemeter-client-74754b6cd9-49r6n\" (UID: \"fb3d3e48-51f2-4503-8026-79787ee98754\") " pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" Oct 30 22:08:33.273346 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.273302 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/fb3d3e48-51f2-4503-8026-79787ee98754-secret-telemeter-client\") pod \"telemeter-client-74754b6cd9-49r6n\" (UID: \"fb3d3e48-51f2-4503-8026-79787ee98754\") " pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" Oct 30 22:08:33.273346 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.273306 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:33.274290 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.273417 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:33.274290 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.273442 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-config\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:33.274290 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.273454 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c373c767-d743-4819-be7c-4c9c830eb9db-metrics-tls\") pod \"dns-default-pl4x2\" (UID: \"c373c767-d743-4819-be7c-4c9c830eb9db\") " pod="openshift-dns/dns-default-pl4x2" Oct 30 22:08:33.274290 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.273501 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:33.274290 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.273556 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/fb3d3e48-51f2-4503-8026-79787ee98754-secret-telemeter-client\") pod \"telemeter-client-74754b6cd9-49r6n\" (UID: \"fb3d3e48-51f2-4503-8026-79787ee98754\") " pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" Oct 30 22:08:33.274290 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.273578 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-multus-daemon-config\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.274290 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.273610 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:33.274290 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.273689 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-config\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:33.274290 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.273715 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-service-ca\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:33.274290 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.273737 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:33.274290 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.273804 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-run\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:08:33.274290 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.273842 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/df6b6cae-173c-461c-9195-4dce90bb6656-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-6krqs\" (UID: \"df6b6cae-173c-461c-9195-4dce90bb6656\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6krqs" Oct 30 22:08:33.274290 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.273858 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/11464051-0aa4-4324-b2c7-7ea2096bbed6-serviceca\") pod \"node-ca-dbtlp\" (UID: \"11464051-0aa4-4324-b2c7-7ea2096bbed6\") " pod="openshift-image-registry/node-ca-dbtlp" Oct 30 22:08:33.274290 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:33.273886 5949 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered Oct 30 22:08:33.274290 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.273897 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9c0d0893-e78b-48f3-bbe8-9f7d0381f42e-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-29lx9\" (UID: \"9c0d0893-e78b-48f3-bbe8-9f7d0381f42e\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-29lx9" Oct 30 22:08:33.274290 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:33.274015 5949 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-service-ca podName:94283f0e-a719-4152-aea8-e219f7bb7eef nodeName:}" failed. No retries permitted until 2026-10-30 22:08:33.773998909 +0000 UTC m=+1.737807462 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-service-ca") pod "console-6d4659d6dd-rb87s" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef") : object "openshift-console"/"service-ca" not registered Oct 30 22:08:33.274290 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.274021 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:33.274290 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.274034 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/df6b6cae-173c-461c-9195-4dce90bb6656-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-6krqs\" (UID: \"df6b6cae-173c-461c-9195-4dce90bb6656\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6krqs" Oct 30 22:08:33.274290 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.274052 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e5779bc8-21ad-465e-8749-8f7e9afdc03f-sys-fs\") pod \"aws-ebs-csi-driver-node-c69jc\" (UID: \"e5779bc8-21ad-465e-8749-8f7e9afdc03f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" Oct 30 22:08:33.274290 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.274171 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cj545\" (UniqueName: \"kubernetes.io/projected/c373c767-d743-4819-be7c-4c9c830eb9db-kube-api-access-cj545\") pod \"dns-default-pl4x2\" (UID: \"c373c767-d743-4819-be7c-4c9c830eb9db\") " pod="openshift-dns/dns-default-pl4x2" Oct 30 22:08:33.274290 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.274204 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-hostroot\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.274290 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.274246 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e129700d-0583-4316-bcee-431c57b0effa-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-xs8x9\" (UID: \"e129700d-0583-4316-bcee-431c57b0effa\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-xs8x9" Oct 30 22:08:33.274290 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.274268 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9c0d0893-e78b-48f3-bbe8-9f7d0381f42e-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-29lx9\" (UID: \"9c0d0893-e78b-48f3-bbe8-9f7d0381f42e\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-29lx9" Oct 30 22:08:33.274290 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.274277 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:33.274290 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.274300 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.274427 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.274486 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-oauth-config\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.274501 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.274514 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e5779bc8-21ad-465e-8749-8f7e9afdc03f-device-dir\") pod \"aws-ebs-csi-driver-node-c69jc\" (UID: \"e5779bc8-21ad-465e-8749-8f7e9afdc03f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.274554 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.274574 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8r24v\" (UniqueName: \"kubernetes.io/projected/5dfaf077-3637-4cc9-8681-466c4e2cc5f4-kube-api-access-8r24v\") pod \"insights-runtime-extractor-pn4kg\" (UID: \"5dfaf077-3637-4cc9-8681-466c4e2cc5f4\") " pod="openshift-insights/insights-runtime-extractor-pn4kg" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.274580 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e129700d-0583-4316-bcee-431c57b0effa-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-xs8x9\" (UID: \"e129700d-0583-4316-bcee-431c57b0effa\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-xs8x9" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.274603 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:33.274622 5949 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.274632 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c9381d26-e2af-4730-8b4e-06945c59585c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v75vr\" (UID: \"c9381d26-e2af-4730-8b4e-06945c59585c\") " pod="openshift-multus/multus-additional-cni-plugins-v75vr" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.274661 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-host-run-multus-certs\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:33.274686 5949 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-oauth-config podName:94283f0e-a719-4152-aea8-e219f7bb7eef nodeName:}" failed. No retries permitted until 2026-10-30 22:08:33.774670581 +0000 UTC m=+1.738479175 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-oauth-config") pod "console-6d4659d6dd-rb87s" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef") : object "openshift-console"/"console-oauth-config" not registered Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.274715 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.274730 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/074eeae7-932c-4642-8fea-671898135ab1-ovnkube-script-lib\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.274749 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/df6b6cae-173c-461c-9195-4dce90bb6656-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-6krqs\" (UID: \"df6b6cae-173c-461c-9195-4dce90bb6656\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6krqs" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.274764 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45l5v\" (UniqueName: \"kubernetes.io/projected/df6b6cae-173c-461c-9195-4dce90bb6656-kube-api-access-45l5v\") pod \"kube-state-metrics-5dc8d7d89b-6krqs\" (UID: \"df6b6cae-173c-461c-9195-4dce90bb6656\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6krqs" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.274783 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-systemd-units\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.274798 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-host-slash\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.274819 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-host-run-ovn-kubernetes\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.274819 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.274834 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/68935e75-a4f0-4a54-9094-4f12b4a2dc39-node-exporter-tls\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.274851 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/68935e75-a4f0-4a54-9094-4f12b4a2dc39-node-exporter-accelerators-collector-config\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.274866 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-oauth-serving-cert\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.274872 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.274881 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-etc-kubernetes\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.274899 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtbtv\" (UniqueName: \"kubernetes.io/projected/fb3d3e48-51f2-4503-8026-79787ee98754-kube-api-access-jtbtv\") pod \"telemeter-client-74754b6cd9-49r6n\" (UID: \"fb3d3e48-51f2-4503-8026-79787ee98754\") " pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.274914 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-run-systemd\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.274928 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kxhz\" (UniqueName: \"kubernetes.io/projected/074eeae7-932c-4642-8fea-671898135ab1-kube-api-access-8kxhz\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.274945 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f64980b7-13c0-40c3-aa3f-277916f24518-cert\") pod \"ingress-canary-9mkk7\" (UID: \"f64980b7-13c0-40c3-aa3f-277916f24518\") " pod="openshift-ingress-canary/ingress-canary-9mkk7" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.274960 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2kkt\" (UniqueName: \"kubernetes.io/projected/94283f0e-a719-4152-aea8-e219f7bb7eef-kube-api-access-k2kkt\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.274975 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf5c4\" (UniqueName: \"kubernetes.io/projected/c9381d26-e2af-4730-8b4e-06945c59585c-kube-api-access-xf5c4\") pod \"multus-additional-cni-plugins-v75vr\" (UID: \"c9381d26-e2af-4730-8b4e-06945c59585c\") " pod="openshift-multus/multus-additional-cni-plugins-v75vr" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.274992 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-host-run-k8s-cni-cncf-io\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.275026 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e5779bc8-21ad-465e-8749-8f7e9afdc03f-etc-selinux\") pod \"aws-ebs-csi-driver-node-c69jc\" (UID: \"e5779bc8-21ad-465e-8749-8f7e9afdc03f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.274847 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c9381d26-e2af-4730-8b4e-06945c59585c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v75vr\" (UID: \"c9381d26-e2af-4730-8b4e-06945c59585c\") " pod="openshift-multus/multus-additional-cni-plugins-v75vr" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.275038 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/074eeae7-932c-4642-8fea-671898135ab1-ovnkube-script-lib\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.275049 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/68935e75-a4f0-4a54-9094-4f12b4a2dc39-node-exporter-accelerators-collector-config\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.274922 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/df6b6cae-173c-461c-9195-4dce90bb6656-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-6krqs\" (UID: \"df6b6cae-173c-461c-9195-4dce90bb6656\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6krqs" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.275081 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:33.275102 5949 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.275117 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.275146 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.275172 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c9381d26-e2af-4730-8b4e-06945c59585c-cni-binary-copy\") pod \"multus-additional-cni-plugins-v75vr\" (UID: \"c9381d26-e2af-4730-8b4e-06945c59585c\") " pod="openshift-multus/multus-additional-cni-plugins-v75vr" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.275203 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-etc-systemd\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.275249 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2prlk\" (UniqueName: \"kubernetes.io/projected/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-kube-api-access-2prlk\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.275309 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.275324 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-host-run-netns\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.275351 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-var-lib-openvswitch\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.275379 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/68935e75-a4f0-4a54-9094-4f12b4a2dc39-node-exporter-tls\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.275382 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zxs5\" (UniqueName: \"kubernetes.io/projected/e129700d-0583-4316-bcee-431c57b0effa-kube-api-access-5zxs5\") pod \"openshift-state-metrics-7bd7489957-xs8x9\" (UID: \"e129700d-0583-4316-bcee-431c57b0effa\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-xs8x9" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.275412 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c9381d26-e2af-4730-8b4e-06945c59585c-system-cni-dir\") pod \"multus-additional-cni-plugins-v75vr\" (UID: \"c9381d26-e2af-4730-8b4e-06945c59585c\") " pod="openshift-multus/multus-additional-cni-plugins-v75vr" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.275439 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c9381d26-e2af-4730-8b4e-06945c59585c-cnibin\") pod \"multus-additional-cni-plugins-v75vr\" (UID: \"c9381d26-e2af-4730-8b4e-06945c59585c\") " pod="openshift-multus/multus-additional-cni-plugins-v75vr" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.275497 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c9381d26-e2af-4730-8b4e-06945c59585c-cni-binary-copy\") pod \"multus-additional-cni-plugins-v75vr\" (UID: \"c9381d26-e2af-4730-8b4e-06945c59585c\") " pod="openshift-multus/multus-additional-cni-plugins-v75vr" Oct 30 22:08:33.275473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.275502 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c9381d26-e2af-4730-8b4e-06945c59585c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v75vr\" (UID: \"c9381d26-e2af-4730-8b4e-06945c59585c\") " pod="openshift-multus/multus-additional-cni-plugins-v75vr" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.275517 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.275531 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9ac70a73-f7f0-4b3e-bf76-09869caefea0-audit-log\") pod \"metrics-server-f89f5d58d-95dk2\" (UID: \"9ac70a73-f7f0-4b3e-bf76-09869caefea0\") " pod="openshift-monitoring/metrics-server-f89f5d58d-95dk2" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.275558 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-cni-binary-copy\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.275583 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65c2ef45-7211-4828-86bc-c9d7af39ec66-metrics-certs\") pod \"network-metrics-daemon-l7kkv\" (UID: \"65c2ef45-7211-4828-86bc-c9d7af39ec66\") " pod="openshift-multus/network-metrics-daemon-l7kkv" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.275614 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f6cc017b-3f50-423a-93da-e4515336a2bc-konnectivity-ca\") pod \"konnectivity-agent-zlbzf\" (UID: \"f6cc017b-3f50-423a-93da-e4515336a2bc\") " pod="kube-system/konnectivity-agent-zlbzf" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:33.275674 5949 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-oauth-serving-cert podName:94283f0e-a719-4152-aea8-e219f7bb7eef nodeName:}" failed. No retries permitted until 2026-10-30 22:08:33.77565757 +0000 UTC m=+1.739466132 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-oauth-serving-cert") pod "console-6d4659d6dd-rb87s" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef") : object "openshift-console"/"oauth-serving-cert" not registered Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.275708 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ac70a73-f7f0-4b3e-bf76-09869caefea0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-f89f5d58d-95dk2\" (UID: \"9ac70a73-f7f0-4b3e-bf76-09869caefea0\") " pod="openshift-monitoring/metrics-server-f89f5d58d-95dk2" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.275720 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.275739 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-sys\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.275754 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f64980b7-13c0-40c3-aa3f-277916f24518-cert\") pod \"ingress-canary-9mkk7\" (UID: \"f64980b7-13c0-40c3-aa3f-277916f24518\") " pod="openshift-ingress-canary/ingress-canary-9mkk7" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.275766 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-run-ovn\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.275790 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6b042dce-1fff-41a2-83dd-ae18f3098207-hosts-file\") pod \"node-resolver-pnr6k\" (UID: \"6b042dce-1fff-41a2-83dd-ae18f3098207\") " pod="openshift-dns/node-resolver-pnr6k" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.275810 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65c2ef45-7211-4828-86bc-c9d7af39ec66-metrics-certs\") pod \"network-metrics-daemon-l7kkv\" (UID: \"65c2ef45-7211-4828-86bc-c9d7af39ec66\") " pod="openshift-multus/network-metrics-daemon-l7kkv" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.275820 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrwhs\" (UniqueName: \"kubernetes.io/projected/11464051-0aa4-4324-b2c7-7ea2096bbed6-kube-api-access-nrwhs\") pod \"node-ca-dbtlp\" (UID: \"11464051-0aa4-4324-b2c7-7ea2096bbed6\") " pod="openshift-image-registry/node-ca-dbtlp" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.275860 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f6cc017b-3f50-423a-93da-e4515336a2bc-konnectivity-ca\") pod \"konnectivity-agent-zlbzf\" (UID: \"f6cc017b-3f50-423a-93da-e4515336a2bc\") " pod="kube-system/konnectivity-agent-zlbzf" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.275919 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9ac70a73-f7f0-4b3e-bf76-09869caefea0-audit-log\") pod \"metrics-server-f89f5d58d-95dk2\" (UID: \"9ac70a73-f7f0-4b3e-bf76-09869caefea0\") " pod="openshift-monitoring/metrics-server-f89f5d58d-95dk2" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.275925 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-cni-binary-copy\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.275971 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.276048 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.276052 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-host\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.276091 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3913a2c9-e88c-4f49-bc63-02b1fa543c38-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.276102 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ac70a73-f7f0-4b3e-bf76-09869caefea0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-f89f5d58d-95dk2\" (UID: \"9ac70a73-f7f0-4b3e-bf76-09869caefea0\") " pod="openshift-monitoring/metrics-server-f89f5d58d-95dk2" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.276119 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94283f0e-a719-4152-aea8-e219f7bb7eef-tmp\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.276239 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e5779bc8-21ad-465e-8749-8f7e9afdc03f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c69jc\" (UID: \"e5779bc8-21ad-465e-8749-8f7e9afdc03f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.276271 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94283f0e-a719-4152-aea8-e219f7bb7eef-tmp\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.276277 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/1b444016-092c-472b-979a-4a7b16271e27-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.276294 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b444016-092c-472b-979a-4a7b16271e27-tmp\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.276321 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3913a2c9-e88c-4f49-bc63-02b1fa543c38-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.276395 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9ac70a73-f7f0-4b3e-bf76-09869caefea0-secret-metrics-server-client-certs\") pod \"metrics-server-f89f5d58d-95dk2\" (UID: \"9ac70a73-f7f0-4b3e-bf76-09869caefea0\") " pod="openshift-monitoring/metrics-server-f89f5d58d-95dk2" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.276420 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/1b444016-092c-472b-979a-4a7b16271e27-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.276448 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-etc-modprobe-d\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.276476 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b444016-092c-472b-979a-4a7b16271e27-tmp\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.276558 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-host-kubelet\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.276590 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-host-cni-netd\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.276619 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-serving-cert\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.276626 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.276662 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-console-config\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:33.276692 5949 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.276698 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9ac70a73-f7f0-4b3e-bf76-09869caefea0-secret-metrics-server-client-certs\") pod \"metrics-server-f89f5d58d-95dk2\" (UID: \"9ac70a73-f7f0-4b3e-bf76-09869caefea0\") " pod="openshift-monitoring/metrics-server-f89f5d58d-95dk2" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.276709 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:33.276747 5949 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-serving-cert podName:94283f0e-a719-4152-aea8-e219f7bb7eef nodeName:}" failed. No retries permitted until 2026-10-30 22:08:33.776733213 +0000 UTC m=+1.740541776 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-serving-cert") pod "console-6d4659d6dd-rb87s" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef") : object "openshift-console"/"console-serving-cert" not registered Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:33.276771 5949 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.276781 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-etc-sysctl-d\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.276813 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpz88\" (UniqueName: \"kubernetes.io/projected/65c2ef45-7211-4828-86bc-c9d7af39ec66-kube-api-access-dpz88\") pod \"network-metrics-daemon-l7kkv\" (UID: \"65c2ef45-7211-4828-86bc-c9d7af39ec66\") " pod="openshift-multus/network-metrics-daemon-l7kkv" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:33.276877 5949 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-console-config podName:94283f0e-a719-4152-aea8-e219f7bb7eef nodeName:}" failed. No retries permitted until 2026-10-30 22:08:33.776862674 +0000 UTC m=+1.740671224 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-console-config") pod "console-6d4659d6dd-rb87s" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef") : object "openshift-console"/"console-config" not registered Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.276897 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/074eeae7-932c-4642-8fea-671898135ab1-ovn-node-metrics-cert\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.276915 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.277027 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxr72\" (UniqueName: \"kubernetes.io/projected/9c0d0893-e78b-48f3-bbe8-9f7d0381f42e-kube-api-access-hxr72\") pod \"monitoring-plugin-77f4f76594-29lx9\" (UID: \"9c0d0893-e78b-48f3-bbe8-9f7d0381f42e\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-29lx9" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.277098 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5dfaf077-3637-4cc9-8681-466c4e2cc5f4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pn4kg\" (UID: \"5dfaf077-3637-4cc9-8681-466c4e2cc5f4\") " pod="openshift-insights/insights-runtime-extractor-pn4kg" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.277120 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-config-volume\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.277136 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-web-config\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.277151 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/074eeae7-932c-4642-8fea-671898135ab1-ovn-node-metrics-cert\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.277170 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5bb239d5-4f0e-4dd6-b58b-5f3b9e014895-host-slash\") pod \"iptables-alerter-b68bn\" (UID: \"5bb239d5-4f0e-4dd6-b58b-5f3b9e014895\") " pod="openshift-network-operator/iptables-alerter-b68bn" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.277190 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1b444016-092c-472b-979a-4a7b16271e27-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.277262 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9ac70a73-f7f0-4b3e-bf76-09869caefea0-metrics-server-audit-profiles\") pod \"metrics-server-f89f5d58d-95dk2\" (UID: \"9ac70a73-f7f0-4b3e-bf76-09869caefea0\") " pod="openshift-monitoring/metrics-server-f89f5d58d-95dk2" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.277304 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsvp8\" (UniqueName: \"kubernetes.io/projected/3913a2c9-e88c-4f49-bc63-02b1fa543c38-kube-api-access-qsvp8\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.277345 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5dfaf077-3637-4cc9-8681-466c4e2cc5f4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pn4kg\" (UID: \"5dfaf077-3637-4cc9-8681-466c4e2cc5f4\") " pod="openshift-insights/insights-runtime-extractor-pn4kg" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.277345 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/fb3d3e48-51f2-4503-8026-79787ee98754-telemeter-client-tls\") pod \"telemeter-client-74754b6cd9-49r6n\" (UID: \"fb3d3e48-51f2-4503-8026-79787ee98754\") " pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.277391 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-config-volume\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.277427 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-var-lib-kubelet\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.277455 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-web-config\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.277460 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.277503 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.277531 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9ac70a73-f7f0-4b3e-bf76-09869caefea0-metrics-server-audit-profiles\") pod \"metrics-server-f89f5d58d-95dk2\" (UID: \"9ac70a73-f7f0-4b3e-bf76-09869caefea0\") " pod="openshift-monitoring/metrics-server-f89f5d58d-95dk2" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.277577 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e5779bc8-21ad-465e-8749-8f7e9afdc03f-socket-dir\") pod \"aws-ebs-csi-driver-node-c69jc\" (UID: \"e5779bc8-21ad-465e-8749-8f7e9afdc03f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.277593 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1b444016-092c-472b-979a-4a7b16271e27-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.277609 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c373c767-d743-4819-be7c-4c9c830eb9db-tmp-dir\") pod \"dns-default-pl4x2\" (UID: \"c373c767-d743-4819-be7c-4c9c830eb9db\") " pod="openshift-dns/dns-default-pl4x2" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.277613 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/fb3d3e48-51f2-4503-8026-79787ee98754-telemeter-client-tls\") pod \"telemeter-client-74754b6cd9-49r6n\" (UID: \"fb3d3e48-51f2-4503-8026-79787ee98754\") " pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" Oct 30 22:08:33.277583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.277627 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9ac70a73-f7f0-4b3e-bf76-09869caefea0-secret-metrics-server-tls\") pod \"metrics-server-f89f5d58d-95dk2\" (UID: \"9ac70a73-f7f0-4b3e-bf76-09869caefea0\") " pod="openshift-monitoring/metrics-server-f89f5d58d-95dk2" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.277634 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.277648 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.277653 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-tmp\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.277685 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/074eeae7-932c-4642-8fea-671898135ab1-ovnkube-config\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.277703 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c373c767-d743-4819-be7c-4c9c830eb9db-config-volume\") pod \"dns-default-pl4x2\" (UID: \"c373c767-d743-4819-be7c-4c9c830eb9db\") " pod="openshift-dns/dns-default-pl4x2" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.277742 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5dfaf077-3637-4cc9-8681-466c4e2cc5f4-data-volume\") pod \"insights-runtime-extractor-pn4kg\" (UID: \"5dfaf077-3637-4cc9-8681-466c4e2cc5f4\") " pod="openshift-insights/insights-runtime-extractor-pn4kg" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.277777 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c373c767-d743-4819-be7c-4c9c830eb9db-tmp-dir\") pod \"dns-default-pl4x2\" (UID: \"c373c767-d743-4819-be7c-4c9c830eb9db\") " pod="openshift-dns/dns-default-pl4x2" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.277782 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5dfaf077-3637-4cc9-8681-466c4e2cc5f4-data-volume\") pod \"insights-runtime-extractor-pn4kg\" (UID: \"5dfaf077-3637-4cc9-8681-466c4e2cc5f4\") " pod="openshift-insights/insights-runtime-extractor-pn4kg" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.277802 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9ac70a73-f7f0-4b3e-bf76-09869caefea0-secret-metrics-server-tls\") pod \"metrics-server-f89f5d58d-95dk2\" (UID: \"9ac70a73-f7f0-4b3e-bf76-09869caefea0\") " pod="openshift-monitoring/metrics-server-f89f5d58d-95dk2" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.277806 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knh4d\" (UniqueName: \"kubernetes.io/projected/5bb239d5-4f0e-4dd6-b58b-5f3b9e014895-kube-api-access-knh4d\") pod \"iptables-alerter-b68bn\" (UID: \"5bb239d5-4f0e-4dd6-b58b-5f3b9e014895\") " pod="openshift-network-operator/iptables-alerter-b68bn" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.277822 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-tmp\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.277826 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb3d3e48-51f2-4503-8026-79787ee98754-serving-certs-ca-bundle\") pod \"telemeter-client-74754b6cd9-49r6n\" (UID: \"fb3d3e48-51f2-4503-8026-79787ee98754\") " pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.277851 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.277948 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/074eeae7-932c-4642-8fea-671898135ab1-ovnkube-config\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.277960 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c373c767-d743-4819-be7c-4c9c830eb9db-config-volume\") pod \"dns-default-pl4x2\" (UID: \"c373c767-d743-4819-be7c-4c9c830eb9db\") " pod="openshift-dns/dns-default-pl4x2" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.277964 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb3d3e48-51f2-4503-8026-79787ee98754-serving-certs-ca-bundle\") pod \"telemeter-client-74754b6cd9-49r6n\" (UID: \"fb3d3e48-51f2-4503-8026-79787ee98754\") " pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.277975 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-node-log\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.278019 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjk5b\" (UniqueName: \"kubernetes.io/projected/f64980b7-13c0-40c3-aa3f-277916f24518-kube-api-access-tjk5b\") pod \"ingress-canary-9mkk7\" (UID: \"f64980b7-13c0-40c3-aa3f-277916f24518\") " pod="openshift-ingress-canary/ingress-canary-9mkk7" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.278048 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cd46f3e8-c752-4eed-a65d-07e060bf32b4-original-pull-secret\") pod \"global-pull-secret-syncer-h9v2l\" (UID: \"cd46f3e8-c752-4eed-a65d-07e060bf32b4\") " pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.278119 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.278154 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-etc-kubernetes\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.278198 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/df6b6cae-173c-461c-9195-4dce90bb6656-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-6krqs\" (UID: \"df6b6cae-173c-461c-9195-4dce90bb6656\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6krqs" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.278238 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/df6b6cae-173c-461c-9195-4dce90bb6656-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-6krqs\" (UID: \"df6b6cae-173c-461c-9195-4dce90bb6656\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6krqs" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.278257 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-lib-modules\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.278279 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-etc-tuned\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.278302 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-etc-openvswitch\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.278340 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-run-openvswitch\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.278358 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-etc-sysctl-conf\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.278373 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-multus-socket-dir-parent\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.278386 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-etc-tuned\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.278393 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-host-var-lib-cni-bin\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.278417 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/df6b6cae-173c-461c-9195-4dce90bb6656-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-6krqs\" (UID: \"df6b6cae-173c-461c-9195-4dce90bb6656\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6krqs" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.278419 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.278484 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-web-config\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.278553 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/df6b6cae-173c-461c-9195-4dce90bb6656-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-6krqs\" (UID: \"df6b6cae-173c-461c-9195-4dce90bb6656\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6krqs" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.278545 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.278615 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.278675 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/68935e75-a4f0-4a54-9094-4f12b4a2dc39-node-exporter-textfile\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.278681 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cd46f3e8-c752-4eed-a65d-07e060bf32b4-original-pull-secret\") pod \"global-pull-secret-syncer-h9v2l\" (UID: \"cd46f3e8-c752-4eed-a65d-07e060bf32b4\") " pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.278722 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e5779bc8-21ad-465e-8749-8f7e9afdc03f-registration-dir\") pod \"aws-ebs-csi-driver-node-c69jc\" (UID: \"e5779bc8-21ad-465e-8749-8f7e9afdc03f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.278744 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-web-config\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.278775 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/68935e75-a4f0-4a54-9094-4f12b4a2dc39-node-exporter-textfile\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.278810 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.278848 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpcht\" (UniqueName: \"kubernetes.io/projected/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-kube-api-access-bpcht\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.278874 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.278942 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1b444016-092c-472b-979a-4a7b16271e27-secret-grpc-tls\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.278981 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.279016 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzvcp\" (UniqueName: \"kubernetes.io/projected/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-kube-api-access-hzvcp\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.279035 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e129700d-0583-4316-bcee-431c57b0effa-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-xs8x9\" (UID: \"e129700d-0583-4316-bcee-431c57b0effa\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-xs8x9" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.279051 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e129700d-0583-4316-bcee-431c57b0effa-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-xs8x9\" (UID: \"e129700d-0583-4316-bcee-431c57b0effa\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-xs8x9" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.279074 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fb3d3e48-51f2-4503-8026-79787ee98754-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-74754b6cd9-49r6n\" (UID: \"fb3d3e48-51f2-4503-8026-79787ee98754\") " pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.279254 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e129700d-0583-4316-bcee-431c57b0effa-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-xs8x9\" (UID: \"e129700d-0583-4316-bcee-431c57b0effa\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-xs8x9" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.279258 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-etc-sysconfig\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.279268 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e129700d-0583-4316-bcee-431c57b0effa-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-xs8x9\" (UID: \"e129700d-0583-4316-bcee-431c57b0effa\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-xs8x9" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.279275 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fb3d3e48-51f2-4503-8026-79787ee98754-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-74754b6cd9-49r6n\" (UID: \"fb3d3e48-51f2-4503-8026-79787ee98754\") " pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.279265 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1b444016-092c-472b-979a-4a7b16271e27-secret-grpc-tls\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.279298 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-cnibin\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.279375 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-host-run-netns\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.279394 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-host-cni-bin\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.279411 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/68935e75-a4f0-4a54-9094-4f12b4a2dc39-sys\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:33.279954 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.279429 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5dfaf077-3637-4cc9-8681-466c4e2cc5f4-crio-socket\") pod \"insights-runtime-extractor-pn4kg\" (UID: \"5dfaf077-3637-4cc9-8681-466c4e2cc5f4\") " pod="openshift-insights/insights-runtime-extractor-pn4kg" Oct 30 22:08:33.282118 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.282102 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:33.292605 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.292587 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6fv5\" (UniqueName: \"kubernetes.io/projected/68935e75-a4f0-4a54-9094-4f12b4a2dc39-kube-api-access-z6fv5\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:33.293472 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.293451 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gvkg\" (UniqueName: \"kubernetes.io/projected/3155dfa9-0dc2-4256-9e16-490b3721702d-kube-api-access-2gvkg\") pod \"network-check-target-knzz5\" (UID: \"3155dfa9-0dc2-4256-9e16-490b3721702d\") " pod="openshift-network-diagnostics/network-check-target-knzz5" Oct 30 22:08:33.293521 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.293512 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9862\" (UniqueName: \"kubernetes.io/projected/1b444016-092c-472b-979a-4a7b16271e27-kube-api-access-j9862\") pod \"thanos-querier-5ddf98bd44-xfj64\" (UID: \"1b444016-092c-472b-979a-4a7b16271e27\") " pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:33.293663 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.293648 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxdbc\" (UniqueName: \"kubernetes.io/projected/6b042dce-1fff-41a2-83dd-ae18f3098207-kube-api-access-qxdbc\") pod \"node-resolver-pnr6k\" (UID: \"6b042dce-1fff-41a2-83dd-ae18f3098207\") " pod="openshift-dns/node-resolver-pnr6k" Oct 30 22:08:33.294023 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.294003 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfsr2\" (UniqueName: \"kubernetes.io/projected/9ac70a73-f7f0-4b3e-bf76-09869caefea0-kube-api-access-bfsr2\") pod \"metrics-server-f89f5d58d-95dk2\" (UID: \"9ac70a73-f7f0-4b3e-bf76-09869caefea0\") " pod="openshift-monitoring/metrics-server-f89f5d58d-95dk2" Oct 30 22:08:33.295028 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.295010 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cj545\" (UniqueName: \"kubernetes.io/projected/c373c767-d743-4819-be7c-4c9c830eb9db-kube-api-access-cj545\") pod \"dns-default-pl4x2\" (UID: \"c373c767-d743-4819-be7c-4c9c830eb9db\") " pod="openshift-dns/dns-default-pl4x2" Oct 30 22:08:33.296435 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:33.296419 5949 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered Oct 30 22:08:33.296497 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:33.296444 5949 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered Oct 30 22:08:33.296497 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:33.296458 5949 projected.go:196] Error preparing data for projected volume kube-api-access-k2kkt for pod openshift-console/console-6d4659d6dd-rb87s: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Oct 30 22:08:33.296497 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.296464 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8r24v\" (UniqueName: \"kubernetes.io/projected/5dfaf077-3637-4cc9-8681-466c4e2cc5f4-kube-api-access-8r24v\") pod \"insights-runtime-extractor-pn4kg\" (UID: \"5dfaf077-3637-4cc9-8681-466c4e2cc5f4\") " pod="openshift-insights/insights-runtime-extractor-pn4kg" Oct 30 22:08:33.296640 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:33.296522 5949 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/94283f0e-a719-4152-aea8-e219f7bb7eef-kube-api-access-k2kkt podName:94283f0e-a719-4152-aea8-e219f7bb7eef nodeName:}" failed. No retries permitted until 2026-10-30 22:08:33.796507897 +0000 UTC m=+1.760316467 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-k2kkt" (UniqueName: "kubernetes.io/projected/94283f0e-a719-4152-aea8-e219f7bb7eef-kube-api-access-k2kkt") pod "console-6d4659d6dd-rb87s" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Oct 30 22:08:33.296966 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.296947 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kxhz\" (UniqueName: \"kubernetes.io/projected/074eeae7-932c-4642-8fea-671898135ab1-kube-api-access-8kxhz\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.297104 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.297087 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrwhs\" (UniqueName: \"kubernetes.io/projected/11464051-0aa4-4324-b2c7-7ea2096bbed6-kube-api-access-nrwhs\") pod \"node-ca-dbtlp\" (UID: \"11464051-0aa4-4324-b2c7-7ea2096bbed6\") " pod="openshift-image-registry/node-ca-dbtlp" Oct 30 22:08:33.297288 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.297273 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zxs5\" (UniqueName: \"kubernetes.io/projected/e129700d-0583-4316-bcee-431c57b0effa-kube-api-access-5zxs5\") pod \"openshift-state-metrics-7bd7489957-xs8x9\" (UID: \"e129700d-0583-4316-bcee-431c57b0effa\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-xs8x9" Oct 30 22:08:33.298375 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.298356 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2prlk\" (UniqueName: \"kubernetes.io/projected/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-kube-api-access-2prlk\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:08:33.298449 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.298402 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf5c4\" (UniqueName: \"kubernetes.io/projected/c9381d26-e2af-4730-8b4e-06945c59585c-kube-api-access-xf5c4\") pod \"multus-additional-cni-plugins-v75vr\" (UID: \"c9381d26-e2af-4730-8b4e-06945c59585c\") " pod="openshift-multus/multus-additional-cni-plugins-v75vr" Oct 30 22:08:33.298685 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.298664 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtbtv\" (UniqueName: \"kubernetes.io/projected/fb3d3e48-51f2-4503-8026-79787ee98754-kube-api-access-jtbtv\") pod \"telemeter-client-74754b6cd9-49r6n\" (UID: \"fb3d3e48-51f2-4503-8026-79787ee98754\") " pod="openshift-monitoring/telemeter-client-74754b6cd9-49r6n" Oct 30 22:08:33.298728 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.298712 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-45l5v\" (UniqueName: \"kubernetes.io/projected/df6b6cae-173c-461c-9195-4dce90bb6656-kube-api-access-45l5v\") pod \"kube-state-metrics-5dc8d7d89b-6krqs\" (UID: \"df6b6cae-173c-461c-9195-4dce90bb6656\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-6krqs" Oct 30 22:08:33.299328 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.299309 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-knh4d\" (UniqueName: \"kubernetes.io/projected/5bb239d5-4f0e-4dd6-b58b-5f3b9e014895-kube-api-access-knh4d\") pod \"iptables-alerter-b68bn\" (UID: \"5bb239d5-4f0e-4dd6-b58b-5f3b9e014895\") " pod="openshift-network-operator/iptables-alerter-b68bn" Oct 30 22:08:33.299860 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.299842 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpz88\" (UniqueName: \"kubernetes.io/projected/65c2ef45-7211-4828-86bc-c9d7af39ec66-kube-api-access-dpz88\") pod \"network-metrics-daemon-l7kkv\" (UID: \"65c2ef45-7211-4828-86bc-c9d7af39ec66\") " pod="openshift-multus/network-metrics-daemon-l7kkv" Oct 30 22:08:33.300052 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.300038 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjk5b\" (UniqueName: \"kubernetes.io/projected/f64980b7-13c0-40c3-aa3f-277916f24518-kube-api-access-tjk5b\") pod \"ingress-canary-9mkk7\" (UID: \"f64980b7-13c0-40c3-aa3f-277916f24518\") " pod="openshift-ingress-canary/ingress-canary-9mkk7" Oct 30 22:08:33.301341 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.301323 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzvcp\" (UniqueName: \"kubernetes.io/projected/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-kube-api-access-hzvcp\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.302074 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.302054 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpcht\" (UniqueName: \"kubernetes.io/projected/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-kube-api-access-bpcht\") pod \"alertmanager-main-0\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:33.302381 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.302363 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsvp8\" (UniqueName: \"kubernetes.io/projected/3913a2c9-e88c-4f49-bc63-02b1fa543c38-kube-api-access-qsvp8\") pod \"prometheus-k8s-0\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:33.302510 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.302493 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxr72\" (UniqueName: \"kubernetes.io/projected/9c0d0893-e78b-48f3-bbe8-9f7d0381f42e-kube-api-access-hxr72\") pod \"monitoring-plugin-77f4f76594-29lx9\" (UID: \"9c0d0893-e78b-48f3-bbe8-9f7d0381f42e\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-29lx9" Oct 30 22:08:33.303023 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.303006 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7f59\" (UniqueName: \"kubernetes.io/projected/e5779bc8-21ad-465e-8749-8f7e9afdc03f-kube-api-access-h7f59\") pod \"aws-ebs-csi-driver-node-c69jc\" (UID: \"e5779bc8-21ad-465e-8749-8f7e9afdc03f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" Oct 30 22:08:33.379939 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.379877 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-hostroot\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.379939 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.379905 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.380079 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.379955 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e5779bc8-21ad-465e-8749-8f7e9afdc03f-device-dir\") pod \"aws-ebs-csi-driver-node-c69jc\" (UID: \"e5779bc8-21ad-465e-8749-8f7e9afdc03f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" Oct 30 22:08:33.380079 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.379981 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-host-run-multus-certs\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.380079 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.379997 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.380079 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.380007 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-systemd-units\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.380079 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.380022 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e5779bc8-21ad-465e-8749-8f7e9afdc03f-device-dir\") pod \"aws-ebs-csi-driver-node-c69jc\" (UID: \"e5779bc8-21ad-465e-8749-8f7e9afdc03f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" Oct 30 22:08:33.380079 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.380038 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-host-slash\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.380079 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.380042 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-systemd-units\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.380079 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.380064 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-host-run-ovn-kubernetes\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.380079 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.380070 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-host-slash\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.380079 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.380081 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-host-run-multus-certs\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.380554 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.379981 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-hostroot\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.380554 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.380105 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-host-run-ovn-kubernetes\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.380554 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.380115 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-etc-kubernetes\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.380554 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.380136 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-run-systemd\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.380554 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.380161 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-host-run-k8s-cni-cncf-io\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.380554 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.380176 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e5779bc8-21ad-465e-8749-8f7e9afdc03f-etc-selinux\") pod \"aws-ebs-csi-driver-node-c69jc\" (UID: \"e5779bc8-21ad-465e-8749-8f7e9afdc03f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" Oct 30 22:08:33.380554 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.380196 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-run-systemd\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.380554 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.380200 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-etc-systemd\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:08:33.380554 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.380237 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-etc-kubernetes\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.380554 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.380245 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-host-run-k8s-cni-cncf-io\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.380554 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.380255 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-host-run-netns\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.380554 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.380264 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-etc-systemd\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:08:33.380554 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.380278 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e5779bc8-21ad-465e-8749-8f7e9afdc03f-etc-selinux\") pod \"aws-ebs-csi-driver-node-c69jc\" (UID: \"e5779bc8-21ad-465e-8749-8f7e9afdc03f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" Oct 30 22:08:33.380554 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.380284 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-host-run-netns\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.380554 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.380294 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-var-lib-openvswitch\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.380554 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.380335 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c9381d26-e2af-4730-8b4e-06945c59585c-system-cni-dir\") pod \"multus-additional-cni-plugins-v75vr\" (UID: \"c9381d26-e2af-4730-8b4e-06945c59585c\") " pod="openshift-multus/multus-additional-cni-plugins-v75vr" Oct 30 22:08:33.380554 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.380359 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c9381d26-e2af-4730-8b4e-06945c59585c-cnibin\") pod \"multus-additional-cni-plugins-v75vr\" (UID: \"c9381d26-e2af-4730-8b4e-06945c59585c\") " pod="openshift-multus/multus-additional-cni-plugins-v75vr" Oct 30 22:08:33.380554 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.380362 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-var-lib-openvswitch\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.380554 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.380382 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c9381d26-e2af-4730-8b4e-06945c59585c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v75vr\" (UID: \"c9381d26-e2af-4730-8b4e-06945c59585c\") " pod="openshift-multus/multus-additional-cni-plugins-v75vr" Oct 30 22:08:33.380554 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.380404 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-sys\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:08:33.380554 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.380413 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c9381d26-e2af-4730-8b4e-06945c59585c-system-cni-dir\") pod \"multus-additional-cni-plugins-v75vr\" (UID: \"c9381d26-e2af-4730-8b4e-06945c59585c\") " pod="openshift-multus/multus-additional-cni-plugins-v75vr" Oct 30 22:08:33.380554 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.380451 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-run-ovn\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.380554 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.380449 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c9381d26-e2af-4730-8b4e-06945c59585c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v75vr\" (UID: \"c9381d26-e2af-4730-8b4e-06945c59585c\") " pod="openshift-multus/multus-additional-cni-plugins-v75vr" Oct 30 22:08:33.380554 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.380418 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-run-ovn\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.380554 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.380479 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-sys\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:08:33.380554 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.380480 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c9381d26-e2af-4730-8b4e-06945c59585c-cnibin\") pod \"multus-additional-cni-plugins-v75vr\" (UID: \"c9381d26-e2af-4730-8b4e-06945c59585c\") " pod="openshift-multus/multus-additional-cni-plugins-v75vr" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.380949 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6b042dce-1fff-41a2-83dd-ae18f3098207-hosts-file\") pod \"node-resolver-pnr6k\" (UID: \"6b042dce-1fff-41a2-83dd-ae18f3098207\") " pod="openshift-dns/node-resolver-pnr6k" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.380979 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-host\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381004 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e5779bc8-21ad-465e-8749-8f7e9afdc03f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c69jc\" (UID: \"e5779bc8-21ad-465e-8749-8f7e9afdc03f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381030 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-etc-modprobe-d\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381044 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-host\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381043 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6b042dce-1fff-41a2-83dd-ae18f3098207-hosts-file\") pod \"node-resolver-pnr6k\" (UID: \"6b042dce-1fff-41a2-83dd-ae18f3098207\") " pod="openshift-dns/node-resolver-pnr6k" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381055 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-host-kubelet\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381065 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e5779bc8-21ad-465e-8749-8f7e9afdc03f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c69jc\" (UID: \"e5779bc8-21ad-465e-8749-8f7e9afdc03f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381078 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-host-cni-netd\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381108 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-host-kubelet\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381142 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-host-cni-netd\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381147 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-etc-modprobe-d\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381181 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-etc-sysctl-d\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381207 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5bb239d5-4f0e-4dd6-b58b-5f3b9e014895-host-slash\") pod \"iptables-alerter-b68bn\" (UID: \"5bb239d5-4f0e-4dd6-b58b-5f3b9e014895\") " pod="openshift-network-operator/iptables-alerter-b68bn" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381247 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-var-lib-kubelet\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381270 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e5779bc8-21ad-465e-8749-8f7e9afdc03f-socket-dir\") pod \"aws-ebs-csi-driver-node-c69jc\" (UID: \"e5779bc8-21ad-465e-8749-8f7e9afdc03f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381287 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5bb239d5-4f0e-4dd6-b58b-5f3b9e014895-host-slash\") pod \"iptables-alerter-b68bn\" (UID: \"5bb239d5-4f0e-4dd6-b58b-5f3b9e014895\") " pod="openshift-network-operator/iptables-alerter-b68bn" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381296 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-node-log\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381319 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-etc-kubernetes\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381324 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-var-lib-kubelet\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381332 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e5779bc8-21ad-465e-8749-8f7e9afdc03f-socket-dir\") pod \"aws-ebs-csi-driver-node-c69jc\" (UID: \"e5779bc8-21ad-465e-8749-8f7e9afdc03f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381345 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-lib-modules\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381352 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-etc-sysctl-d\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381369 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-node-log\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381369 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-etc-openvswitch\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381392 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-etc-openvswitch\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381403 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-run-openvswitch\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381425 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-lib-modules\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381375 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-etc-kubernetes\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381426 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-etc-sysctl-conf\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381465 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-multus-socket-dir-parent\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381485 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-host-var-lib-cni-bin\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381465 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-run-openvswitch\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381506 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e5779bc8-21ad-465e-8749-8f7e9afdc03f-registration-dir\") pod \"aws-ebs-csi-driver-node-c69jc\" (UID: \"e5779bc8-21ad-465e-8749-8f7e9afdc03f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381537 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-etc-sysconfig\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381544 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-host-var-lib-cni-bin\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381534 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-multus-socket-dir-parent\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381575 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-etc-sysctl-conf\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381594 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-etc-sysconfig\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381595 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e5779bc8-21ad-465e-8749-8f7e9afdc03f-registration-dir\") pod \"aws-ebs-csi-driver-node-c69jc\" (UID: \"e5779bc8-21ad-465e-8749-8f7e9afdc03f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381609 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-cnibin\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381628 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-host-run-netns\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381638 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-cnibin\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381649 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-host-cni-bin\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381678 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/68935e75-a4f0-4a54-9094-4f12b4a2dc39-sys\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381682 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-host-cni-bin\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381679 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-host-run-netns\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381703 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5dfaf077-3637-4cc9-8681-466c4e2cc5f4-crio-socket\") pod \"insights-runtime-extractor-pn4kg\" (UID: \"5dfaf077-3637-4cc9-8681-466c4e2cc5f4\") " pod="openshift-insights/insights-runtime-extractor-pn4kg" Oct 30 22:08:33.381689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381710 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/68935e75-a4f0-4a54-9094-4f12b4a2dc39-sys\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:33.383192 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381730 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-host-var-lib-kubelet\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.383192 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381741 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5dfaf077-3637-4cc9-8681-466c4e2cc5f4-crio-socket\") pod \"insights-runtime-extractor-pn4kg\" (UID: \"5dfaf077-3637-4cc9-8681-466c4e2cc5f4\") " pod="openshift-insights/insights-runtime-extractor-pn4kg" Oct 30 22:08:33.383192 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381754 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11464051-0aa4-4324-b2c7-7ea2096bbed6-host\") pod \"node-ca-dbtlp\" (UID: \"11464051-0aa4-4324-b2c7-7ea2096bbed6\") " pod="openshift-image-registry/node-ca-dbtlp" Oct 30 22:08:33.383192 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381775 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cd46f3e8-c752-4eed-a65d-07e060bf32b4-kubelet-config\") pod \"global-pull-secret-syncer-h9v2l\" (UID: \"cd46f3e8-c752-4eed-a65d-07e060bf32b4\") " pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:08:33.383192 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381804 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-log-socket\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.383192 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381820 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/68935e75-a4f0-4a54-9094-4f12b4a2dc39-root\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:33.383192 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381820 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-host-var-lib-kubelet\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.383192 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381836 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-multus-cni-dir\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.383192 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381839 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cd46f3e8-c752-4eed-a65d-07e060bf32b4-kubelet-config\") pod \"global-pull-secret-syncer-h9v2l\" (UID: \"cd46f3e8-c752-4eed-a65d-07e060bf32b4\") " pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:08:33.383192 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381837 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11464051-0aa4-4324-b2c7-7ea2096bbed6-host\") pod \"node-ca-dbtlp\" (UID: \"11464051-0aa4-4324-b2c7-7ea2096bbed6\") " pod="openshift-image-registry/node-ca-dbtlp" Oct 30 22:08:33.383192 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381871 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/074eeae7-932c-4642-8fea-671898135ab1-log-socket\") pod \"ovnkube-node-ql6lh\" (UID: \"074eeae7-932c-4642-8fea-671898135ab1\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.383192 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381880 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/68935e75-a4f0-4a54-9094-4f12b4a2dc39-root\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:33.383192 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381895 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-multus-cni-dir\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.383192 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381871 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-system-cni-dir\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.383192 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381951 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cd46f3e8-c752-4eed-a65d-07e060bf32b4-dbus\") pod \"global-pull-secret-syncer-h9v2l\" (UID: \"cd46f3e8-c752-4eed-a65d-07e060bf32b4\") " pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:08:33.383192 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381913 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-system-cni-dir\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.383192 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.381989 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/68935e75-a4f0-4a54-9094-4f12b4a2dc39-node-exporter-wtmp\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:33.383192 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.382018 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-host-var-lib-cni-multus\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.383192 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.382034 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cd46f3e8-c752-4eed-a65d-07e060bf32b4-dbus\") pod \"global-pull-secret-syncer-h9v2l\" (UID: \"cd46f3e8-c752-4eed-a65d-07e060bf32b4\") " pod="kube-system/global-pull-secret-syncer-h9v2l" Oct 30 22:08:33.383192 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.382066 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-multus-conf-dir\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.383192 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.382075 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-host-var-lib-cni-multus\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.383192 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.382106 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/68935e75-a4f0-4a54-9094-4f12b4a2dc39-node-exporter-wtmp\") pod \"node-exporter-s7svn\" (UID: \"68935e75-a4f0-4a54-9094-4f12b4a2dc39\") " pod="openshift-monitoring/node-exporter-s7svn" Oct 30 22:08:33.383192 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.382112 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-run\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:08:33.383192 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.382143 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c1f7e385-c1ea-4fd9-aff3-513ae337b5d8-multus-conf-dir\") pod \"multus-f6888\" (UID: \"c1f7e385-c1ea-4fd9-aff3-513ae337b5d8\") " pod="openshift-multus/multus-f6888" Oct 30 22:08:33.383192 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.382160 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e5779bc8-21ad-465e-8749-8f7e9afdc03f-sys-fs\") pod \"aws-ebs-csi-driver-node-c69jc\" (UID: \"e5779bc8-21ad-465e-8749-8f7e9afdc03f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" Oct 30 22:08:33.383192 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.382155 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44-run\") pod \"tuned-8hv7m\" (UID: \"e74d4847-dcbb-4b6a-9a4e-dc3a49ef7a44\") " pod="openshift-cluster-node-tuning-operator/tuned-8hv7m" Oct 30 22:08:33.383192 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.382197 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e5779bc8-21ad-465e-8749-8f7e9afdc03f-sys-fs\") pod \"aws-ebs-csi-driver-node-c69jc\" (UID: \"e5779bc8-21ad-465e-8749-8f7e9afdc03f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c69jc" Oct 30 22:08:33.704521 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.704485 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:33.786294 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.786260 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-serving-cert\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:33.786294 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.786298 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-console-config\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:33.786494 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.786404 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-trusted-ca-bundle\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:33.786494 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:33.786409 5949 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered Oct 30 22:08:33.786494 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:33.786413 5949 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered Oct 30 22:08:33.786494 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.786440 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-service-ca\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:33.786494 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:33.786482 5949 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-serving-cert podName:94283f0e-a719-4152-aea8-e219f7bb7eef nodeName:}" failed. No retries permitted until 2026-10-30 22:08:34.786464215 +0000 UTC m=+2.750272780 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-serving-cert") pod "console-6d4659d6dd-rb87s" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef") : object "openshift-console"/"console-serving-cert" not registered Oct 30 22:08:33.786494 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:33.786489 5949 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered Oct 30 22:08:33.786758 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:33.786498 5949 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered Oct 30 22:08:33.786758 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.786503 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-oauth-config\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:33.786758 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:33.786544 5949 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-service-ca podName:94283f0e-a719-4152-aea8-e219f7bb7eef nodeName:}" failed. No retries permitted until 2026-10-30 22:08:34.786532753 +0000 UTC m=+2.750341307 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-service-ca") pod "console-6d4659d6dd-rb87s" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef") : object "openshift-console"/"service-ca" not registered Oct 30 22:08:33.786758 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.786560 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-oauth-serving-cert\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:33.786758 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:33.786610 5949 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered Oct 30 22:08:33.786758 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:33.786616 5949 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-console-config podName:94283f0e-a719-4152-aea8-e219f7bb7eef nodeName:}" failed. No retries permitted until 2026-10-30 22:08:34.786610086 +0000 UTC m=+2.750418635 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-console-config") pod "console-6d4659d6dd-rb87s" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef") : object "openshift-console"/"console-config" not registered Oct 30 22:08:33.786758 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:33.786628 5949 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered Oct 30 22:08:33.786758 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:33.786633 5949 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-trusted-ca-bundle podName:94283f0e-a719-4152-aea8-e219f7bb7eef nodeName:}" failed. No retries permitted until 2026-10-30 22:08:34.786627469 +0000 UTC m=+2.750436016 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-trusted-ca-bundle") pod "console-6d4659d6dd-rb87s" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef") : object "openshift-console"/"trusted-ca-bundle" not registered Oct 30 22:08:33.786758 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:33.786715 5949 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-oauth-config podName:94283f0e-a719-4152-aea8-e219f7bb7eef nodeName:}" failed. No retries permitted until 2026-10-30 22:08:34.786696575 +0000 UTC m=+2.750505130 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-oauth-config") pod "console-6d4659d6dd-rb87s" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef") : object "openshift-console"/"console-oauth-config" not registered Oct 30 22:08:33.786758 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:33.786738 5949 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-oauth-serving-cert podName:94283f0e-a719-4152-aea8-e219f7bb7eef nodeName:}" failed. No retries permitted until 2026-10-30 22:08:34.786728801 +0000 UTC m=+2.750537356 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-oauth-serving-cert") pod "console-6d4659d6dd-rb87s" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef") : object "openshift-console"/"oauth-serving-cert" not registered Oct 30 22:08:33.887928 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:33.887901 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2kkt\" (UniqueName: \"kubernetes.io/projected/94283f0e-a719-4152-aea8-e219f7bb7eef-kube-api-access-k2kkt\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:33.888063 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:33.888020 5949 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered Oct 30 22:08:33.888063 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:33.888033 5949 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered Oct 30 22:08:33.888063 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:33.888043 5949 projected.go:196] Error preparing data for projected volume kube-api-access-k2kkt for pod openshift-console/console-6d4659d6dd-rb87s: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Oct 30 22:08:33.888171 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:33.888081 5949 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/94283f0e-a719-4152-aea8-e219f7bb7eef-kube-api-access-k2kkt podName:94283f0e-a719-4152-aea8-e219f7bb7eef nodeName:}" failed. No retries permitted until 2026-10-30 22:08:34.888070095 +0000 UTC m=+2.851878642 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-k2kkt" (UniqueName: "kubernetes.io/projected/94283f0e-a719-4152-aea8-e219f7bb7eef-kube-api-access-k2kkt") pod "console-6d4659d6dd-rb87s" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Oct 30 22:08:34.280881 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:34.280847 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5ddf98bd44-xfj64" Oct 30 22:08:34.440685 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:34.440652 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:34.798546 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:34.798510 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-trusted-ca-bundle\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:34.798709 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:34.798554 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-service-ca\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:34.798709 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:34.798586 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-oauth-config\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:34.798709 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:34.798616 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-oauth-serving-cert\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:34.798709 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:34.798646 5949 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered Oct 30 22:08:34.798709 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:34.798696 5949 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered Oct 30 22:08:34.798961 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:34.798717 5949 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-service-ca podName:94283f0e-a719-4152-aea8-e219f7bb7eef nodeName:}" failed. No retries permitted until 2026-10-30 22:08:36.798702114 +0000 UTC m=+4.762510662 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-service-ca") pod "console-6d4659d6dd-rb87s" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef") : object "openshift-console"/"service-ca" not registered Oct 30 22:08:34.798961 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:34.798653 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-serving-cert\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:34.798961 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:34.798752 5949 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-oauth-serving-cert podName:94283f0e-a719-4152-aea8-e219f7bb7eef nodeName:}" failed. No retries permitted until 2026-10-30 22:08:36.798737426 +0000 UTC m=+4.762545979 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-oauth-serving-cert") pod "console-6d4659d6dd-rb87s" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef") : object "openshift-console"/"oauth-serving-cert" not registered Oct 30 22:08:34.798961 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:34.798752 5949 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered Oct 30 22:08:34.798961 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:34.798771 5949 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered Oct 30 22:08:34.798961 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:34.798797 5949 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered Oct 30 22:08:34.798961 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:34.798776 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-console-config\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:34.798961 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:34.798816 5949 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-oauth-config podName:94283f0e-a719-4152-aea8-e219f7bb7eef nodeName:}" failed. No retries permitted until 2026-10-30 22:08:36.798803118 +0000 UTC m=+4.762611666 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-oauth-config") pod "console-6d4659d6dd-rb87s" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef") : object "openshift-console"/"console-oauth-config" not registered Oct 30 22:08:34.798961 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:34.798835 5949 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-trusted-ca-bundle podName:94283f0e-a719-4152-aea8-e219f7bb7eef nodeName:}" failed. No retries permitted until 2026-10-30 22:08:36.798829566 +0000 UTC m=+4.762638113 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-trusted-ca-bundle") pod "console-6d4659d6dd-rb87s" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef") : object "openshift-console"/"trusted-ca-bundle" not registered Oct 30 22:08:34.798961 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:34.798849 5949 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-console-config podName:94283f0e-a719-4152-aea8-e219f7bb7eef nodeName:}" failed. No retries permitted until 2026-10-30 22:08:36.798842484 +0000 UTC m=+4.762651032 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-console-config") pod "console-6d4659d6dd-rb87s" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef") : object "openshift-console"/"console-config" not registered Oct 30 22:08:34.798961 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:34.798864 5949 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered Oct 30 22:08:34.798961 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:34.798907 5949 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-serving-cert podName:94283f0e-a719-4152-aea8-e219f7bb7eef nodeName:}" failed. No retries permitted until 2026-10-30 22:08:36.798892097 +0000 UTC m=+4.762700644 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-serving-cert") pod "console-6d4659d6dd-rb87s" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef") : object "openshift-console"/"console-serving-cert" not registered Oct 30 22:08:34.899988 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:34.899963 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2kkt\" (UniqueName: \"kubernetes.io/projected/94283f0e-a719-4152-aea8-e219f7bb7eef-kube-api-access-k2kkt\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:34.900124 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:34.900106 5949 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered Oct 30 22:08:34.900124 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:34.900124 5949 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered Oct 30 22:08:34.900202 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:34.900133 5949 projected.go:196] Error preparing data for projected volume kube-api-access-k2kkt for pod openshift-console/console-6d4659d6dd-rb87s: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Oct 30 22:08:34.900202 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:34.900174 5949 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/94283f0e-a719-4152-aea8-e219f7bb7eef-kube-api-access-k2kkt podName:94283f0e-a719-4152-aea8-e219f7bb7eef nodeName:}" failed. No retries permitted until 2026-10-30 22:08:36.900163827 +0000 UTC m=+4.863972379 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-k2kkt" (UniqueName: "kubernetes.io/projected/94283f0e-a719-4152-aea8-e219f7bb7eef-kube-api-access-k2kkt") pod "console-6d4659d6dd-rb87s" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Oct 30 22:08:35.237784 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:35.237754 5949 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 22:08:36.020841 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:36.020812 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-pl4x2" Oct 30 22:08:36.442212 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:36.442180 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-f89f5d58d-95dk2" Oct 30 22:08:36.819985 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:36.819912 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-oauth-config\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:36.819985 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:36.819955 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-oauth-serving-cert\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:36.820263 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:36.819997 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-serving-cert\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:36.820263 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:36.820014 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-console-config\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:36.820263 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:36.820064 5949 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered Oct 30 22:08:36.820263 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:36.820094 5949 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered Oct 30 22:08:36.820263 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:36.820131 5949 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-oauth-config podName:94283f0e-a719-4152-aea8-e219f7bb7eef nodeName:}" failed. No retries permitted until 2026-10-30 22:08:40.820116023 +0000 UTC m=+8.783924607 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-oauth-config") pod "console-6d4659d6dd-rb87s" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef") : object "openshift-console"/"console-oauth-config" not registered Oct 30 22:08:36.820263 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:36.820132 5949 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered Oct 30 22:08:36.820263 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:36.820140 5949 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered Oct 30 22:08:36.820263 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:36.820132 5949 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered Oct 30 22:08:36.820263 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:36.820167 5949 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-oauth-serving-cert podName:94283f0e-a719-4152-aea8-e219f7bb7eef nodeName:}" failed. No retries permitted until 2026-10-30 22:08:40.820152222 +0000 UTC m=+8.783960791 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-oauth-serving-cert") pod "console-6d4659d6dd-rb87s" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef") : object "openshift-console"/"oauth-serving-cert" not registered Oct 30 22:08:36.820263 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:36.820207 5949 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-serving-cert podName:94283f0e-a719-4152-aea8-e219f7bb7eef nodeName:}" failed. No retries permitted until 2026-10-30 22:08:40.820191926 +0000 UTC m=+8.784000491 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-serving-cert") pod "console-6d4659d6dd-rb87s" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef") : object "openshift-console"/"console-serving-cert" not registered Oct 30 22:08:36.820263 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:36.820070 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-trusted-ca-bundle\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:36.820263 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:36.820244 5949 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-trusted-ca-bundle podName:94283f0e-a719-4152-aea8-e219f7bb7eef nodeName:}" failed. No retries permitted until 2026-10-30 22:08:40.820234168 +0000 UTC m=+8.784042721 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-trusted-ca-bundle") pod "console-6d4659d6dd-rb87s" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef") : object "openshift-console"/"trusted-ca-bundle" not registered Oct 30 22:08:36.820263 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:36.820272 5949 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-console-config podName:94283f0e-a719-4152-aea8-e219f7bb7eef nodeName:}" failed. No retries permitted until 2026-10-30 22:08:40.820262027 +0000 UTC m=+8.784070595 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-console-config") pod "console-6d4659d6dd-rb87s" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef") : object "openshift-console"/"console-config" not registered Oct 30 22:08:36.820745 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:36.820321 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-service-ca\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:36.820745 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:36.820401 5949 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered Oct 30 22:08:36.820745 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:36.820444 5949 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-service-ca podName:94283f0e-a719-4152-aea8-e219f7bb7eef nodeName:}" failed. No retries permitted until 2026-10-30 22:08:40.820433369 +0000 UTC m=+8.784241927 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-service-ca") pod "console-6d4659d6dd-rb87s" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef") : object "openshift-console"/"service-ca" not registered Oct 30 22:08:36.921771 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:36.921741 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2kkt\" (UniqueName: \"kubernetes.io/projected/94283f0e-a719-4152-aea8-e219f7bb7eef-kube-api-access-k2kkt\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:36.921921 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:36.921908 5949 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered Oct 30 22:08:36.921978 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:36.921929 5949 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered Oct 30 22:08:36.921978 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:36.921951 5949 projected.go:196] Error preparing data for projected volume kube-api-access-k2kkt for pod openshift-console/console-6d4659d6dd-rb87s: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Oct 30 22:08:36.922035 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:36.922002 5949 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/94283f0e-a719-4152-aea8-e219f7bb7eef-kube-api-access-k2kkt podName:94283f0e-a719-4152-aea8-e219f7bb7eef nodeName:}" failed. No retries permitted until 2026-10-30 22:08:40.92198986 +0000 UTC m=+8.885798421 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-k2kkt" (UniqueName: "kubernetes.io/projected/94283f0e-a719-4152-aea8-e219f7bb7eef-kube-api-access-k2kkt") pod "console-6d4659d6dd-rb87s" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Oct 30 22:08:37.251110 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:37.251083 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:38.118691 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:38.118662 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-29lx9" Oct 30 22:08:40.865369 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:40.865328 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-trusted-ca-bundle\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:40.865369 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:40.865374 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-service-ca\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:40.865898 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:40.865397 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-oauth-config\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:40.865898 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:40.865418 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-oauth-serving-cert\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:40.865898 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:40.865457 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-serving-cert\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:40.865898 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:40.865457 5949 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered Oct 30 22:08:40.865898 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:40.865472 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-console-config\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:40.865898 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:40.865483 5949 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered Oct 30 22:08:40.865898 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:40.865457 5949 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered Oct 30 22:08:40.865898 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:40.865526 5949 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered Oct 30 22:08:40.865898 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:40.865536 5949 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-trusted-ca-bundle podName:94283f0e-a719-4152-aea8-e219f7bb7eef nodeName:}" failed. No retries permitted until 2026-10-30 22:08:48.865514356 +0000 UTC m=+16.829322927 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-trusted-ca-bundle") pod "console-6d4659d6dd-rb87s" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef") : object "openshift-console"/"trusted-ca-bundle" not registered Oct 30 22:08:40.865898 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:40.865558 5949 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-oauth-serving-cert podName:94283f0e-a719-4152-aea8-e219f7bb7eef nodeName:}" failed. No retries permitted until 2026-10-30 22:08:48.865548227 +0000 UTC m=+16.829356788 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-oauth-serving-cert") pod "console-6d4659d6dd-rb87s" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef") : object "openshift-console"/"oauth-serving-cert" not registered Oct 30 22:08:40.865898 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:40.865520 5949 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered Oct 30 22:08:40.865898 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:40.865575 5949 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-console-config podName:94283f0e-a719-4152-aea8-e219f7bb7eef nodeName:}" failed. No retries permitted until 2026-10-30 22:08:48.865564297 +0000 UTC m=+16.829372848 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-console-config") pod "console-6d4659d6dd-rb87s" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef") : object "openshift-console"/"console-config" not registered Oct 30 22:08:40.865898 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:40.865590 5949 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered Oct 30 22:08:40.865898 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:40.865591 5949 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-service-ca podName:94283f0e-a719-4152-aea8-e219f7bb7eef nodeName:}" failed. No retries permitted until 2026-10-30 22:08:48.865582491 +0000 UTC m=+16.829391045 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-service-ca") pod "console-6d4659d6dd-rb87s" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef") : object "openshift-console"/"service-ca" not registered Oct 30 22:08:40.865898 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:40.865659 5949 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-oauth-config podName:94283f0e-a719-4152-aea8-e219f7bb7eef nodeName:}" failed. No retries permitted until 2026-10-30 22:08:48.865646243 +0000 UTC m=+16.829454803 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-oauth-config") pod "console-6d4659d6dd-rb87s" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef") : object "openshift-console"/"console-oauth-config" not registered Oct 30 22:08:40.865898 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:40.865673 5949 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-serving-cert podName:94283f0e-a719-4152-aea8-e219f7bb7eef nodeName:}" failed. No retries permitted until 2026-10-30 22:08:48.865664393 +0000 UTC m=+16.829472943 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-serving-cert") pod "console-6d4659d6dd-rb87s" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef") : object "openshift-console"/"console-serving-cert" not registered Oct 30 22:08:40.966689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:40.966656 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2kkt\" (UniqueName: \"kubernetes.io/projected/94283f0e-a719-4152-aea8-e219f7bb7eef-kube-api-access-k2kkt\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:40.966825 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:40.966806 5949 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered Oct 30 22:08:40.966825 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:40.966822 5949 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered Oct 30 22:08:40.966895 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:40.966833 5949 projected.go:196] Error preparing data for projected volume kube-api-access-k2kkt for pod openshift-console/console-6d4659d6dd-rb87s: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Oct 30 22:08:40.966895 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:40.966880 5949 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/94283f0e-a719-4152-aea8-e219f7bb7eef-kube-api-access-k2kkt podName:94283f0e-a719-4152-aea8-e219f7bb7eef nodeName:}" failed. No retries permitted until 2026-10-30 22:08:48.96686501 +0000 UTC m=+16.930673573 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-k2kkt" (UniqueName: "kubernetes.io/projected/94283f0e-a719-4152-aea8-e219f7bb7eef-kube-api-access-k2kkt") pod "console-6d4659d6dd-rb87s" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Oct 30 22:08:41.102094 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:41.102067 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-knzz5" Oct 30 22:08:45.987124 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:45.987048 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ql6lh" Oct 30 22:08:48.942159 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:48.942120 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-trusted-ca-bundle\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:48.942656 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:48.942166 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-service-ca\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:48.942656 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:48.942194 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-oauth-config\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:48.942656 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:48.942243 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-oauth-serving-cert\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:48.942656 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:48.942264 5949 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered Oct 30 22:08:48.942656 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:48.942304 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-serving-cert\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:48.942656 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:48.942326 5949 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered Oct 30 22:08:48.942656 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:48.942267 5949 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered Oct 30 22:08:48.942656 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:48.942366 5949 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered Oct 30 22:08:48.942656 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:48.942392 5949 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered Oct 30 22:08:48.942656 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:48.942328 5949 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-service-ca podName:94283f0e-a719-4152-aea8-e219f7bb7eef nodeName:}" failed. No retries permitted until 2026-10-30 22:09:04.942314877 +0000 UTC m=+32.906123451 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-service-ca") pod "console-6d4659d6dd-rb87s" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef") : object "openshift-console"/"service-ca" not registered Oct 30 22:08:48.942656 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:48.942437 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-console-config\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:48.942656 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:48.942449 5949 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-serving-cert podName:94283f0e-a719-4152-aea8-e219f7bb7eef nodeName:}" failed. No retries permitted until 2026-10-30 22:09:04.942431001 +0000 UTC m=+32.906239563 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-serving-cert") pod "console-6d4659d6dd-rb87s" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef") : object "openshift-console"/"console-serving-cert" not registered Oct 30 22:08:48.942656 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:48.942473 5949 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-oauth-config podName:94283f0e-a719-4152-aea8-e219f7bb7eef nodeName:}" failed. No retries permitted until 2026-10-30 22:09:04.942462477 +0000 UTC m=+32.906271028 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-oauth-config") pod "console-6d4659d6dd-rb87s" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef") : object "openshift-console"/"console-oauth-config" not registered Oct 30 22:08:48.942656 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:48.942491 5949 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-trusted-ca-bundle podName:94283f0e-a719-4152-aea8-e219f7bb7eef nodeName:}" failed. No retries permitted until 2026-10-30 22:09:04.942482148 +0000 UTC m=+32.906290696 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-trusted-ca-bundle") pod "console-6d4659d6dd-rb87s" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef") : object "openshift-console"/"trusted-ca-bundle" not registered Oct 30 22:08:48.942656 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:48.942501 5949 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered Oct 30 22:08:48.942656 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:48.942507 5949 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-oauth-serving-cert podName:94283f0e-a719-4152-aea8-e219f7bb7eef nodeName:}" failed. No retries permitted until 2026-10-30 22:09:04.942499889 +0000 UTC m=+32.906308440 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-oauth-serving-cert") pod "console-6d4659d6dd-rb87s" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef") : object "openshift-console"/"oauth-serving-cert" not registered Oct 30 22:08:48.942656 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:48.942581 5949 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-console-config podName:94283f0e-a719-4152-aea8-e219f7bb7eef nodeName:}" failed. No retries permitted until 2026-10-30 22:09:04.942565063 +0000 UTC m=+32.906373624 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-console-config") pod "console-6d4659d6dd-rb87s" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef") : object "openshift-console"/"console-config" not registered Oct 30 22:08:49.043304 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:49.043268 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2kkt\" (UniqueName: \"kubernetes.io/projected/94283f0e-a719-4152-aea8-e219f7bb7eef-kube-api-access-k2kkt\") pod \"console-6d4659d6dd-rb87s\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:49.043446 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:49.043430 5949 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered Oct 30 22:08:49.043484 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:49.043453 5949 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered Oct 30 22:08:49.043484 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:49.043466 5949 projected.go:196] Error preparing data for projected volume kube-api-access-k2kkt for pod openshift-console/console-6d4659d6dd-rb87s: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Oct 30 22:08:49.043548 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:49.043508 5949 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/94283f0e-a719-4152-aea8-e219f7bb7eef-kube-api-access-k2kkt podName:94283f0e-a719-4152-aea8-e219f7bb7eef nodeName:}" failed. No retries permitted until 2026-10-30 22:09:05.043493986 +0000 UTC m=+33.007302545 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-k2kkt" (UniqueName: "kubernetes.io/projected/94283f0e-a719-4152-aea8-e219f7bb7eef-kube-api-access-k2kkt") pod "console-6d4659d6dd-rb87s" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Oct 30 22:08:51.161287 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:51.161255 5949 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 30 22:08:51.161698 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:51.161524 5949 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" containerName="alertmanager" containerID="cri-o://1a1dfe2f444e73e6026fc72da31a2f2e1a8bfc7f425e6552ed63ccc85b4e0171" gracePeriod=120 Oct 30 22:08:51.161698 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:51.161573 5949 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" containerName="kube-rbac-proxy-metric" containerID="cri-o://b9059ad19c6204de4394dea117f2dedd2f6e8b4e62972234263b26a3289fd0db" gracePeriod=120 Oct 30 22:08:51.161698 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:51.161610 5949 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" containerName="prom-label-proxy" containerID="cri-o://47d577d4ee928534952f088b26bc3c9b9746e81390f9b59c682ffdb0d7f0c885" gracePeriod=120 Oct 30 22:08:51.161698 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:51.161656 5949 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" containerName="kube-rbac-proxy" containerID="cri-o://5bccb4128b8010d35e1d2fedc4cf0f253a08a9cde42b580c48c64a4a93fe31fe" gracePeriod=120 Oct 30 22:08:51.161884 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:51.161680 5949 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" containerName="config-reloader" containerID="cri-o://94c755efc0aedc9fb07704a28e0cee7df9af8edaa408e8451da92cb56f28f896" gracePeriod=120 Oct 30 22:08:51.161884 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:51.161623 5949 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" containerName="kube-rbac-proxy-web" containerID="cri-o://d0822fed9f2a2c5827ba836a0563738ad56a344412b85653ce9f04f1e25dbb2d" gracePeriod=120 Oct 30 22:08:51.304424 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:51.304400 5949 generic.go:356] "Generic (PLEG): container finished" podID="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" containerID="47d577d4ee928534952f088b26bc3c9b9746e81390f9b59c682ffdb0d7f0c885" exitCode=0 Oct 30 22:08:51.304424 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:51.304421 5949 generic.go:356] "Generic (PLEG): container finished" podID="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" containerID="5bccb4128b8010d35e1d2fedc4cf0f253a08a9cde42b580c48c64a4a93fe31fe" exitCode=0 Oct 30 22:08:51.304554 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:51.304428 5949 generic.go:356] "Generic (PLEG): container finished" podID="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" containerID="94c755efc0aedc9fb07704a28e0cee7df9af8edaa408e8451da92cb56f28f896" exitCode=0 Oct 30 22:08:51.304554 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:51.304434 5949 generic.go:356] "Generic (PLEG): container finished" podID="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" containerID="1a1dfe2f444e73e6026fc72da31a2f2e1a8bfc7f425e6552ed63ccc85b4e0171" exitCode=0 Oct 30 22:08:51.304554 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:51.304457 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"46b06955-7c3e-4a10-b5cd-5c2f4c34859f","Type":"ContainerDied","Data":"47d577d4ee928534952f088b26bc3c9b9746e81390f9b59c682ffdb0d7f0c885"} Oct 30 22:08:51.304554 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:51.304509 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"46b06955-7c3e-4a10-b5cd-5c2f4c34859f","Type":"ContainerDied","Data":"5bccb4128b8010d35e1d2fedc4cf0f253a08a9cde42b580c48c64a4a93fe31fe"} Oct 30 22:08:51.304554 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:51.304536 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"46b06955-7c3e-4a10-b5cd-5c2f4c34859f","Type":"ContainerDied","Data":"94c755efc0aedc9fb07704a28e0cee7df9af8edaa408e8451da92cb56f28f896"} Oct 30 22:08:51.304710 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:51.304559 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"46b06955-7c3e-4a10-b5cd-5c2f4c34859f","Type":"ContainerDied","Data":"1a1dfe2f444e73e6026fc72da31a2f2e1a8bfc7f425e6552ed63ccc85b4e0171"} Oct 30 22:08:52.311379 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.311330 5949 generic.go:356] "Generic (PLEG): container finished" podID="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" containerID="b9059ad19c6204de4394dea117f2dedd2f6e8b4e62972234263b26a3289fd0db" exitCode=0 Oct 30 22:08:52.311379 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.311361 5949 generic.go:356] "Generic (PLEG): container finished" podID="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" containerID="d0822fed9f2a2c5827ba836a0563738ad56a344412b85653ce9f04f1e25dbb2d" exitCode=0 Oct 30 22:08:52.311871 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.311391 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"46b06955-7c3e-4a10-b5cd-5c2f4c34859f","Type":"ContainerDied","Data":"b9059ad19c6204de4394dea117f2dedd2f6e8b4e62972234263b26a3289fd0db"} Oct 30 22:08:52.311871 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.311466 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"46b06955-7c3e-4a10-b5cd-5c2f4c34859f","Type":"ContainerDied","Data":"d0822fed9f2a2c5827ba836a0563738ad56a344412b85653ce9f04f1e25dbb2d"} Oct 30 22:08:52.407106 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.407078 5949 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:52.476928 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.476864 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-web-config\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-web-config\") pod \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " Oct 30 22:08:52.476928 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.476924 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-metrics-client-ca\") pod \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " Oct 30 22:08:52.477116 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.476955 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-config-volume\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-config-volume\") pod \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " Oct 30 22:08:52.477116 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.476978 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-tls-assets\" (UniqueName: \"kubernetes.io/projected/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-tls-assets\") pod \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " Oct 30 22:08:52.477116 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.477016 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-secret-alertmanager-main-tls\") pod \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " Oct 30 22:08:52.477116 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.477052 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-secret-alertmanager-kube-rbac-proxy-web\") pod \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " Oct 30 22:08:52.477116 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.477081 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-secret-alertmanager-kube-rbac-proxy\") pod \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " Oct 30 22:08:52.477392 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.477267 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-cluster-tls-config\") pod \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " Oct 30 22:08:52.477442 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.477399 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-metrics-client-ca" pod "46b06955-7c3e-4a10-b5cd-5c2f4c34859f" (UID: "46b06955-7c3e-4a10-b5cd-5c2f4c34859f"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:08:52.477442 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.477420 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-alertmanager-main-db\") pod \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " Oct 30 22:08:52.477549 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.477483 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-config-out\" (UniqueName: \"kubernetes.io/empty-dir/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-config-out\") pod \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " Oct 30 22:08:52.477549 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.477540 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-alertmanager-trusted-ca-bundle\") pod \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " Oct 30 22:08:52.477651 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.477610 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-kube-api-access-bpcht\" (UniqueName: \"kubernetes.io/projected/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-kube-api-access-bpcht\") pod \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " Oct 30 22:08:52.477651 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.477642 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\" (UID: \"46b06955-7c3e-4a10-b5cd-5c2f4c34859f\") " Oct 30 22:08:52.477995 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.477974 5949 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-metrics-client-ca\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:08:52.478900 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.478324 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-alertmanager-trusted-ca-bundle" pod "46b06955-7c3e-4a10-b5cd-5c2f4c34859f" (UID: "46b06955-7c3e-4a10-b5cd-5c2f4c34859f"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:08:52.478900 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.478722 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-alertmanager-main-db" pod "46b06955-7c3e-4a10-b5cd-5c2f4c34859f" (UID: "46b06955-7c3e-4a10-b5cd-5c2f4c34859f"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 22:08:52.481274 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.481212 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-config-out" pod "46b06955-7c3e-4a10-b5cd-5c2f4c34859f" (UID: "46b06955-7c3e-4a10-b5cd-5c2f4c34859f"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 22:08:52.481540 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.481494 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-secret-alertmanager-kube-rbac-proxy-metric" pod "46b06955-7c3e-4a10-b5cd-5c2f4c34859f" (UID: "46b06955-7c3e-4a10-b5cd-5c2f4c34859f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:08:52.482327 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.482304 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-config-volume" pod "46b06955-7c3e-4a10-b5cd-5c2f4c34859f" (UID: "46b06955-7c3e-4a10-b5cd-5c2f4c34859f"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:08:52.482420 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.482409 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-tls-assets" pod "46b06955-7c3e-4a10-b5cd-5c2f4c34859f" (UID: "46b06955-7c3e-4a10-b5cd-5c2f4c34859f"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 22:08:52.483441 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.483363 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-secret-alertmanager-main-tls" pod "46b06955-7c3e-4a10-b5cd-5c2f4c34859f" (UID: "46b06955-7c3e-4a10-b5cd-5c2f4c34859f"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:08:52.483664 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.483637 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-kube-api-access-bpcht" pod "46b06955-7c3e-4a10-b5cd-5c2f4c34859f" (UID: "46b06955-7c3e-4a10-b5cd-5c2f4c34859f"). InnerVolumeSpecName "kube-api-access-bpcht". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 22:08:52.483774 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.483715 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-secret-alertmanager-kube-rbac-proxy-web" pod "46b06955-7c3e-4a10-b5cd-5c2f4c34859f" (UID: "46b06955-7c3e-4a10-b5cd-5c2f4c34859f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:08:52.483888 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.483752 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-secret-alertmanager-kube-rbac-proxy" pod "46b06955-7c3e-4a10-b5cd-5c2f4c34859f" (UID: "46b06955-7c3e-4a10-b5cd-5c2f4c34859f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:08:52.488748 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.488606 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-cluster-tls-config" pod "46b06955-7c3e-4a10-b5cd-5c2f4c34859f" (UID: "46b06955-7c3e-4a10-b5cd-5c2f4c34859f"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:08:52.494490 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.494469 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-web-config" pod "46b06955-7c3e-4a10-b5cd-5c2f4c34859f" (UID: "46b06955-7c3e-4a10-b5cd-5c2f4c34859f"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:08:52.580230 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.580194 5949 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-cluster-tls-config\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:08:52.580378 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.580240 5949 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-alertmanager-main-db\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:08:52.580378 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.580252 5949 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-config-out\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:08:52.580378 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.580261 5949 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:08:52.580378 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.580270 5949 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bpcht\" (UniqueName: \"kubernetes.io/projected/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-kube-api-access-bpcht\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:08:52.580378 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.580280 5949 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:08:52.580378 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.580289 5949 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-web-config\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:08:52.580378 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.580298 5949 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-config-volume\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:08:52.580378 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.580306 5949 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-tls-assets\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:08:52.580378 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.580315 5949 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-secret-alertmanager-main-tls\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:08:52.580378 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.580324 5949 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:08:52.580378 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:52.580334 5949 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/46b06955-7c3e-4a10-b5cd-5c2f4c34859f-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:08:53.318512 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.318477 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"46b06955-7c3e-4a10-b5cd-5c2f4c34859f","Type":"ContainerDied","Data":"4e612a431c19e72aa48209c41de2f70414134eda7dbfd4fe026be056f9641119"} Oct 30 22:08:53.318512 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.318511 5949 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:53.318990 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.318550 5949 scope.go:122] "RemoveContainer" containerID="47d577d4ee928534952f088b26bc3c9b9746e81390f9b59c682ffdb0d7f0c885" Oct 30 22:08:53.327182 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.327160 5949 scope.go:122] "RemoveContainer" containerID="b9059ad19c6204de4394dea117f2dedd2f6e8b4e62972234263b26a3289fd0db" Oct 30 22:08:53.334203 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.334188 5949 scope.go:122] "RemoveContainer" containerID="5bccb4128b8010d35e1d2fedc4cf0f253a08a9cde42b580c48c64a4a93fe31fe" Oct 30 22:08:53.340689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.340661 5949 scope.go:122] "RemoveContainer" containerID="d0822fed9f2a2c5827ba836a0563738ad56a344412b85653ce9f04f1e25dbb2d" Oct 30 22:08:53.347117 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.347095 5949 scope.go:122] "RemoveContainer" containerID="94c755efc0aedc9fb07704a28e0cee7df9af8edaa408e8451da92cb56f28f896" Oct 30 22:08:53.355649 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.355635 5949 scope.go:122] "RemoveContainer" containerID="1a1dfe2f444e73e6026fc72da31a2f2e1a8bfc7f425e6552ed63ccc85b4e0171" Oct 30 22:08:53.362979 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.362961 5949 scope.go:122] "RemoveContainer" containerID="e599f9a04f1e39f29f36cd26b16b488cf06bbe03c9228c4fda6d8f10e43c0b31" Oct 30 22:08:53.383616 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.383596 5949 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 30 22:08:53.395091 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.395069 5949 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 30 22:08:53.427334 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.427310 5949 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 30 22:08:53.427596 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.427584 5949 cpu_manager.go:414] "containerMap: removing container" podUID="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" containerName="prom-label-proxy" Oct 30 22:08:53.427638 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.427597 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" containerName="prom-label-proxy" Oct 30 22:08:53.427638 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.427606 5949 cpu_manager.go:414] "containerMap: removing container" podUID="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" containerName="kube-rbac-proxy-web" Oct 30 22:08:53.427638 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.427611 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" containerName="kube-rbac-proxy-web" Oct 30 22:08:53.427638 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.427622 5949 cpu_manager.go:414] "containerMap: removing container" podUID="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" containerName="kube-rbac-proxy-metric" Oct 30 22:08:53.427638 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.427627 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" containerName="kube-rbac-proxy-metric" Oct 30 22:08:53.427638 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.427638 5949 cpu_manager.go:414] "containerMap: removing container" podUID="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" containerName="init-config-reloader" Oct 30 22:08:53.427844 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.427642 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" containerName="init-config-reloader" Oct 30 22:08:53.427844 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.427658 5949 cpu_manager.go:414] "containerMap: removing container" podUID="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" containerName="alertmanager" Oct 30 22:08:53.427844 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.427663 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" containerName="alertmanager" Oct 30 22:08:53.427844 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.427670 5949 cpu_manager.go:414] "containerMap: removing container" podUID="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" containerName="config-reloader" Oct 30 22:08:53.427844 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.427674 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" containerName="config-reloader" Oct 30 22:08:53.427844 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.427683 5949 cpu_manager.go:414] "containerMap: removing container" podUID="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" containerName="kube-rbac-proxy" Oct 30 22:08:53.427844 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.427688 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" containerName="kube-rbac-proxy" Oct 30 22:08:53.427844 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.427734 5949 memory_manager.go:365] "RemoveStaleState removing state" podUID="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" containerName="init-config-reloader" Oct 30 22:08:53.427844 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.427743 5949 memory_manager.go:365] "RemoveStaleState removing state" podUID="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" containerName="alertmanager" Oct 30 22:08:53.427844 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.427748 5949 memory_manager.go:365] "RemoveStaleState removing state" podUID="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" containerName="config-reloader" Oct 30 22:08:53.427844 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.427754 5949 memory_manager.go:365] "RemoveStaleState removing state" podUID="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" containerName="kube-rbac-proxy" Oct 30 22:08:53.427844 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.427760 5949 memory_manager.go:365] "RemoveStaleState removing state" podUID="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" containerName="prom-label-proxy" Oct 30 22:08:53.427844 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.427765 5949 memory_manager.go:365] "RemoveStaleState removing state" podUID="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" containerName="kube-rbac-proxy-web" Oct 30 22:08:53.427844 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.427771 5949 memory_manager.go:365] "RemoveStaleState removing state" podUID="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" containerName="kube-rbac-proxy-metric" Oct 30 22:08:53.431096 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.431078 5949 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:53.453940 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.453922 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Oct 30 22:08:53.454072 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.453994 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Oct 30 22:08:53.454126 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.454098 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Oct 30 22:08:53.454126 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.454099 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Oct 30 22:08:53.454187 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.454135 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Oct 30 22:08:53.454187 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.454181 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Oct 30 22:08:53.454347 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.454333 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Oct 30 22:08:53.454453 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.454436 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Oct 30 22:08:53.454487 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.454436 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-5s2rl\"" Oct 30 22:08:53.466040 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.465999 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Oct 30 22:08:53.474487 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.474471 5949 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 30 22:08:53.499989 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.499964 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/da6d4cc7-1aa7-4a66-8902-d7ded08379cc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"da6d4cc7-1aa7-4a66-8902-d7ded08379cc\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:53.500088 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.499995 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/da6d4cc7-1aa7-4a66-8902-d7ded08379cc-config-volume\") pod \"alertmanager-main-0\" (UID: \"da6d4cc7-1aa7-4a66-8902-d7ded08379cc\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:53.500139 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.500121 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/da6d4cc7-1aa7-4a66-8902-d7ded08379cc-tls-assets\") pod \"alertmanager-main-0\" (UID: \"da6d4cc7-1aa7-4a66-8902-d7ded08379cc\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:53.500180 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.500165 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96skc\" (UniqueName: \"kubernetes.io/projected/da6d4cc7-1aa7-4a66-8902-d7ded08379cc-kube-api-access-96skc\") pod \"alertmanager-main-0\" (UID: \"da6d4cc7-1aa7-4a66-8902-d7ded08379cc\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:53.500267 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.500252 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/da6d4cc7-1aa7-4a66-8902-d7ded08379cc-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"da6d4cc7-1aa7-4a66-8902-d7ded08379cc\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:53.500343 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.500328 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/da6d4cc7-1aa7-4a66-8902-d7ded08379cc-config-out\") pod \"alertmanager-main-0\" (UID: \"da6d4cc7-1aa7-4a66-8902-d7ded08379cc\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:53.500381 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.500360 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/da6d4cc7-1aa7-4a66-8902-d7ded08379cc-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"da6d4cc7-1aa7-4a66-8902-d7ded08379cc\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:53.500414 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.500381 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/da6d4cc7-1aa7-4a66-8902-d7ded08379cc-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"da6d4cc7-1aa7-4a66-8902-d7ded08379cc\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:53.500414 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.500395 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/da6d4cc7-1aa7-4a66-8902-d7ded08379cc-web-config\") pod \"alertmanager-main-0\" (UID: \"da6d4cc7-1aa7-4a66-8902-d7ded08379cc\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:53.500472 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.500445 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/da6d4cc7-1aa7-4a66-8902-d7ded08379cc-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"da6d4cc7-1aa7-4a66-8902-d7ded08379cc\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:53.500521 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.500507 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da6d4cc7-1aa7-4a66-8902-d7ded08379cc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"da6d4cc7-1aa7-4a66-8902-d7ded08379cc\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:53.500557 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.500537 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da6d4cc7-1aa7-4a66-8902-d7ded08379cc-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"da6d4cc7-1aa7-4a66-8902-d7ded08379cc\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:53.500589 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.500556 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/da6d4cc7-1aa7-4a66-8902-d7ded08379cc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"da6d4cc7-1aa7-4a66-8902-d7ded08379cc\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:53.602017 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.601988 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/da6d4cc7-1aa7-4a66-8902-d7ded08379cc-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"da6d4cc7-1aa7-4a66-8902-d7ded08379cc\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:53.602017 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.602021 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/da6d4cc7-1aa7-4a66-8902-d7ded08379cc-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"da6d4cc7-1aa7-4a66-8902-d7ded08379cc\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:53.602311 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.602037 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/da6d4cc7-1aa7-4a66-8902-d7ded08379cc-web-config\") pod \"alertmanager-main-0\" (UID: \"da6d4cc7-1aa7-4a66-8902-d7ded08379cc\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:53.602311 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.602136 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/da6d4cc7-1aa7-4a66-8902-d7ded08379cc-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"da6d4cc7-1aa7-4a66-8902-d7ded08379cc\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:53.602311 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.602164 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da6d4cc7-1aa7-4a66-8902-d7ded08379cc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"da6d4cc7-1aa7-4a66-8902-d7ded08379cc\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:53.602311 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.602192 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da6d4cc7-1aa7-4a66-8902-d7ded08379cc-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"da6d4cc7-1aa7-4a66-8902-d7ded08379cc\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:53.602311 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.602238 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/da6d4cc7-1aa7-4a66-8902-d7ded08379cc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"da6d4cc7-1aa7-4a66-8902-d7ded08379cc\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:53.602311 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.602296 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/da6d4cc7-1aa7-4a66-8902-d7ded08379cc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"da6d4cc7-1aa7-4a66-8902-d7ded08379cc\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:53.602622 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.602327 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/da6d4cc7-1aa7-4a66-8902-d7ded08379cc-config-volume\") pod \"alertmanager-main-0\" (UID: \"da6d4cc7-1aa7-4a66-8902-d7ded08379cc\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:53.602622 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.602405 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/da6d4cc7-1aa7-4a66-8902-d7ded08379cc-tls-assets\") pod \"alertmanager-main-0\" (UID: \"da6d4cc7-1aa7-4a66-8902-d7ded08379cc\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:53.602622 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.602442 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96skc\" (UniqueName: \"kubernetes.io/projected/da6d4cc7-1aa7-4a66-8902-d7ded08379cc-kube-api-access-96skc\") pod \"alertmanager-main-0\" (UID: \"da6d4cc7-1aa7-4a66-8902-d7ded08379cc\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:53.602622 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.602452 5949 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Oct 30 22:08:53.602622 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.602487 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/da6d4cc7-1aa7-4a66-8902-d7ded08379cc-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"da6d4cc7-1aa7-4a66-8902-d7ded08379cc\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:53.602622 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.602541 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/da6d4cc7-1aa7-4a66-8902-d7ded08379cc-config-out\") pod \"alertmanager-main-0\" (UID: \"da6d4cc7-1aa7-4a66-8902-d7ded08379cc\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:53.603107 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.603021 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da6d4cc7-1aa7-4a66-8902-d7ded08379cc-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"da6d4cc7-1aa7-4a66-8902-d7ded08379cc\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:53.603231 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.603121 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da6d4cc7-1aa7-4a66-8902-d7ded08379cc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"da6d4cc7-1aa7-4a66-8902-d7ded08379cc\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:53.603505 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.603482 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/da6d4cc7-1aa7-4a66-8902-d7ded08379cc-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"da6d4cc7-1aa7-4a66-8902-d7ded08379cc\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:53.605327 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.605301 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/da6d4cc7-1aa7-4a66-8902-d7ded08379cc-web-config\") pod \"alertmanager-main-0\" (UID: \"da6d4cc7-1aa7-4a66-8902-d7ded08379cc\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:53.605417 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.605337 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/da6d4cc7-1aa7-4a66-8902-d7ded08379cc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"da6d4cc7-1aa7-4a66-8902-d7ded08379cc\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:53.605630 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.605611 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/da6d4cc7-1aa7-4a66-8902-d7ded08379cc-config-out\") pod \"alertmanager-main-0\" (UID: \"da6d4cc7-1aa7-4a66-8902-d7ded08379cc\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:53.605812 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.605792 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/da6d4cc7-1aa7-4a66-8902-d7ded08379cc-config-volume\") pod \"alertmanager-main-0\" (UID: \"da6d4cc7-1aa7-4a66-8902-d7ded08379cc\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:53.605859 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.605844 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/da6d4cc7-1aa7-4a66-8902-d7ded08379cc-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"da6d4cc7-1aa7-4a66-8902-d7ded08379cc\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:53.605948 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.605927 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/da6d4cc7-1aa7-4a66-8902-d7ded08379cc-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"da6d4cc7-1aa7-4a66-8902-d7ded08379cc\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:53.606261 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.606242 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/da6d4cc7-1aa7-4a66-8902-d7ded08379cc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"da6d4cc7-1aa7-4a66-8902-d7ded08379cc\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:53.606586 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.606571 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/da6d4cc7-1aa7-4a66-8902-d7ded08379cc-tls-assets\") pod \"alertmanager-main-0\" (UID: \"da6d4cc7-1aa7-4a66-8902-d7ded08379cc\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:53.607496 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.607474 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/da6d4cc7-1aa7-4a66-8902-d7ded08379cc-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"da6d4cc7-1aa7-4a66-8902-d7ded08379cc\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:53.627100 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.627074 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96skc\" (UniqueName: \"kubernetes.io/projected/da6d4cc7-1aa7-4a66-8902-d7ded08379cc-kube-api-access-96skc\") pod \"alertmanager-main-0\" (UID: \"da6d4cc7-1aa7-4a66-8902-d7ded08379cc\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:53.747731 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:53.747663 5949 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:54.069043 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:54.068959 5949 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 30 22:08:54.072541 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:54.072511 5949 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda6d4cc7_1aa7_4a66_8902_d7ded08379cc.slice/crio-ade17d7fa24ee76b405e02ff034fc025f0bafcaa9871edad90e46dc283776eee WatchSource:0}: Error finding container ade17d7fa24ee76b405e02ff034fc025f0bafcaa9871edad90e46dc283776eee: Status 404 returned error can't find the container with id ade17d7fa24ee76b405e02ff034fc025f0bafcaa9871edad90e46dc283776eee Oct 30 22:08:54.160233 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:54.160191 5949 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="46b06955-7c3e-4a10-b5cd-5c2f4c34859f" path="/var/lib/kubelet/pods/46b06955-7c3e-4a10-b5cd-5c2f4c34859f/volumes" Oct 30 22:08:54.323916 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:54.323835 5949 generic.go:356] "Generic (PLEG): container finished" podID="da6d4cc7-1aa7-4a66-8902-d7ded08379cc" containerID="5ebd45d7dbe309c1e721d102d1050ed0a6c587aa58a0353833ff66ccd4f9af69" exitCode=0 Oct 30 22:08:54.323916 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:54.323878 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da6d4cc7-1aa7-4a66-8902-d7ded08379cc","Type":"ContainerDied","Data":"5ebd45d7dbe309c1e721d102d1050ed0a6c587aa58a0353833ff66ccd4f9af69"} Oct 30 22:08:54.324330 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:54.323940 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da6d4cc7-1aa7-4a66-8902-d7ded08379cc","Type":"ContainerStarted","Data":"ade17d7fa24ee76b405e02ff034fc025f0bafcaa9871edad90e46dc283776eee"} Oct 30 22:08:55.331742 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:55.331709 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da6d4cc7-1aa7-4a66-8902-d7ded08379cc","Type":"ContainerStarted","Data":"5cc9111772617c949560288b3904fd9b8931163ced2a0215e88045d97eddcfe8"} Oct 30 22:08:55.332121 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:55.331814 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da6d4cc7-1aa7-4a66-8902-d7ded08379cc","Type":"ContainerStarted","Data":"b1f7533b48b40fec9cb9c629361336bf67e06f2f2a645922d99f51ab7b1d9892"} Oct 30 22:08:55.332121 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:55.331843 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da6d4cc7-1aa7-4a66-8902-d7ded08379cc","Type":"ContainerStarted","Data":"47b13feab228de89f672c62f7c0e1fe45d853602f84ba273e4d79cf90896542e"} Oct 30 22:08:55.332121 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:55.331873 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da6d4cc7-1aa7-4a66-8902-d7ded08379cc","Type":"ContainerStarted","Data":"60f73cac120fd8abb56df9bbeb214a3694115135b98162d186cc1af34cd0b00e"} Oct 30 22:08:55.332121 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:55.331913 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da6d4cc7-1aa7-4a66-8902-d7ded08379cc","Type":"ContainerStarted","Data":"e17c4786c0dab53822e92daeed4c8f1cef2e0489935872d3cad3e9a2e7c24430"} Oct 30 22:08:55.332121 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:55.331941 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"da6d4cc7-1aa7-4a66-8902-d7ded08379cc","Type":"ContainerStarted","Data":"e4f4a20442ca3f972065377c3e31c895294c4887a971137b11ccc9bb602f2da2"} Oct 30 22:08:55.336957 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:55.336928 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:08:55.397291 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:55.397239 5949 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.382684667 podStartE2EDuration="2.39720624s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.521573ms" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:55.396074451 +0000 UTC m=+23.359883021" watchObservedRunningTime="2026-10-30 22:08:55.39720624 +0000 UTC m=+23.361014804" Oct 30 22:08:55.792097 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:55.792061 5949 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 30 22:08:55.792549 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:55.792443 5949 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3913a2c9-e88c-4f49-bc63-02b1fa543c38" containerName="kube-rbac-proxy" containerID="cri-o://e65b0800081ec131f68d9c0efae55c42ac26fdd495967642fa18f904d0840737" gracePeriod=600 Oct 30 22:08:55.792549 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:55.792452 5949 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3913a2c9-e88c-4f49-bc63-02b1fa543c38" containerName="config-reloader" containerID="cri-o://6df39d2515c9a702cf3a1d64f1cc0a8a9e056bbb0fa98cf0ebaad9dad3096a05" gracePeriod=600 Oct 30 22:08:55.792549 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:55.792464 5949 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3913a2c9-e88c-4f49-bc63-02b1fa543c38" containerName="kube-rbac-proxy-web" containerID="cri-o://911a2c5672e542f3af43a336f66776e42858c381f94b8a1fe92b1ca4b805c5f5" gracePeriod=600 Oct 30 22:08:55.792549 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:55.792476 5949 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3913a2c9-e88c-4f49-bc63-02b1fa543c38" containerName="prometheus" containerID="cri-o://36bd1a5bed3b82469994cb7d98f25114091efa66af5fe091e3037500ac0d995b" gracePeriod=600 Oct 30 22:08:55.793172 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:55.792456 5949 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3913a2c9-e88c-4f49-bc63-02b1fa543c38" containerName="thanos-sidecar" containerID="cri-o://056a499f0455153c6c6409a71b6292bf1c957cd5d23143a835b0e63819b58859" gracePeriod=600 Oct 30 22:08:55.793172 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:55.792453 5949 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3913a2c9-e88c-4f49-bc63-02b1fa543c38" containerName="kube-rbac-proxy-thanos" containerID="cri-o://b2e637f872238fab5e1353aa253e2d1ec3b076550b69a82adb4d137d1af394d4" gracePeriod=600 Oct 30 22:08:56.039889 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.039866 5949 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.126444 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.126372 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-trusted-ca-bundle\") pod \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " Oct 30 22:08:56.126444 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.126413 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-k8s-db\") pod \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " Oct 30 22:08:56.126645 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.126448 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " Oct 30 22:08:56.126645 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.126476 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-configmap-kubelet-serving-ca-bundle\") pod \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " Oct 30 22:08:56.126645 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.126601 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-thanos-prometheus-http-client-file\") pod \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " Oct 30 22:08:56.126875 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.126662 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-k8s-rulefiles-2\") pod \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " Oct 30 22:08:56.126875 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.126699 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-prometheus-k8s-tls\") pod \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " Oct 30 22:08:56.126875 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.126740 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-k8s-rulefiles-0\") pod \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " Oct 30 22:08:56.126875 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.126780 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-kube-rbac-proxy\") pod \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " Oct 30 22:08:56.126875 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.126809 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-web-config\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-web-config\") pod \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " Oct 30 22:08:56.126875 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.126843 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-configmap-metrics-client-ca\") pod \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " Oct 30 22:08:56.126875 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.126842 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-configmap-kubelet-serving-ca-bundle" pod "3913a2c9-e88c-4f49-bc63-02b1fa543c38" (UID: "3913a2c9-e88c-4f49-bc63-02b1fa543c38"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:08:56.126875 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.126875 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3913a2c9-e88c-4f49-bc63-02b1fa543c38-tls-assets\" (UniqueName: \"kubernetes.io/projected/3913a2c9-e88c-4f49-bc63-02b1fa543c38-tls-assets\") pod \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " Oct 30 22:08:56.127458 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.126879 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-trusted-ca-bundle" pod "3913a2c9-e88c-4f49-bc63-02b1fa543c38" (UID: "3913a2c9-e88c-4f49-bc63-02b1fa543c38"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:08:56.127458 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.126911 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3913a2c9-e88c-4f49-bc63-02b1fa543c38-kube-api-access-qsvp8\" (UniqueName: \"kubernetes.io/projected/3913a2c9-e88c-4f49-bc63-02b1fa543c38-kube-api-access-qsvp8\") pod \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " Oct 30 22:08:56.127458 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.126943 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-config\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-config\") pod \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " Oct 30 22:08:56.127458 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.126978 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-grpc-tls\") pod \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " Oct 30 22:08:56.127458 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.127009 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " Oct 30 22:08:56.127458 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.127038 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-k8s-rulefiles-1\") pod \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " Oct 30 22:08:56.127458 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.127091 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-configmap-serving-certs-ca-bundle\") pod \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " Oct 30 22:08:56.127458 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.127130 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-metrics-client-certs\") pod \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " Oct 30 22:08:56.127458 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.127187 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3913a2c9-e88c-4f49-bc63-02b1fa543c38-config-out\" (UniqueName: \"kubernetes.io/empty-dir/3913a2c9-e88c-4f49-bc63-02b1fa543c38-config-out\") pod \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\" (UID: \"3913a2c9-e88c-4f49-bc63-02b1fa543c38\") " Oct 30 22:08:56.127458 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.127434 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-k8s-rulefiles-2" pod "3913a2c9-e88c-4f49-bc63-02b1fa543c38" (UID: "3913a2c9-e88c-4f49-bc63-02b1fa543c38"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:08:56.127999 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.127617 5949 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:08:56.127999 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.127639 5949 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:08:56.127999 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.127655 5949 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-trusted-ca-bundle\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:08:56.128987 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.128847 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-k8s-rulefiles-0" pod "3913a2c9-e88c-4f49-bc63-02b1fa543c38" (UID: "3913a2c9-e88c-4f49-bc63-02b1fa543c38"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:08:56.129281 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.129194 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-configmap-metrics-client-ca" pod "3913a2c9-e88c-4f49-bc63-02b1fa543c38" (UID: "3913a2c9-e88c-4f49-bc63-02b1fa543c38"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:08:56.129579 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.129529 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-prometheus-k8s-thanos-sidecar-tls" pod "3913a2c9-e88c-4f49-bc63-02b1fa543c38" (UID: "3913a2c9-e88c-4f49-bc63-02b1fa543c38"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:08:56.129656 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.129605 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-k8s-rulefiles-1" pod "3913a2c9-e88c-4f49-bc63-02b1fa543c38" (UID: "3913a2c9-e88c-4f49-bc63-02b1fa543c38"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:08:56.129656 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.129631 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-k8s-db" pod "3913a2c9-e88c-4f49-bc63-02b1fa543c38" (UID: "3913a2c9-e88c-4f49-bc63-02b1fa543c38"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 22:08:56.129820 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.129794 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-thanos-prometheus-http-client-file" pod "3913a2c9-e88c-4f49-bc63-02b1fa543c38" (UID: "3913a2c9-e88c-4f49-bc63-02b1fa543c38"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:08:56.129982 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.129958 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-configmap-serving-certs-ca-bundle" pod "3913a2c9-e88c-4f49-bc63-02b1fa543c38" (UID: "3913a2c9-e88c-4f49-bc63-02b1fa543c38"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:08:56.131055 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.131020 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-prometheus-k8s-tls" pod "3913a2c9-e88c-4f49-bc63-02b1fa543c38" (UID: "3913a2c9-e88c-4f49-bc63-02b1fa543c38"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:08:56.131413 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.131379 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3913a2c9-e88c-4f49-bc63-02b1fa543c38-tls-assets" pod "3913a2c9-e88c-4f49-bc63-02b1fa543c38" (UID: "3913a2c9-e88c-4f49-bc63-02b1fa543c38"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 22:08:56.132137 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.132100 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-metrics-client-certs" pod "3913a2c9-e88c-4f49-bc63-02b1fa543c38" (UID: "3913a2c9-e88c-4f49-bc63-02b1fa543c38"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:08:56.132305 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.132282 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3913a2c9-e88c-4f49-bc63-02b1fa543c38-config-out" pod "3913a2c9-e88c-4f49-bc63-02b1fa543c38" (UID: "3913a2c9-e88c-4f49-bc63-02b1fa543c38"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 22:08:56.132493 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.132469 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-config" pod "3913a2c9-e88c-4f49-bc63-02b1fa543c38" (UID: "3913a2c9-e88c-4f49-bc63-02b1fa543c38"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:08:56.132575 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.132554 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-kube-rbac-proxy" pod "3913a2c9-e88c-4f49-bc63-02b1fa543c38" (UID: "3913a2c9-e88c-4f49-bc63-02b1fa543c38"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:08:56.133106 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.133079 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-prometheus-k8s-kube-rbac-proxy-web" pod "3913a2c9-e88c-4f49-bc63-02b1fa543c38" (UID: "3913a2c9-e88c-4f49-bc63-02b1fa543c38"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:08:56.133620 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.133586 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3913a2c9-e88c-4f49-bc63-02b1fa543c38-kube-api-access-qsvp8" pod "3913a2c9-e88c-4f49-bc63-02b1fa543c38" (UID: "3913a2c9-e88c-4f49-bc63-02b1fa543c38"). InnerVolumeSpecName "kube-api-access-qsvp8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 22:08:56.133798 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.133774 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-grpc-tls" pod "3913a2c9-e88c-4f49-bc63-02b1fa543c38" (UID: "3913a2c9-e88c-4f49-bc63-02b1fa543c38"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:08:56.141560 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.141539 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-web-config" pod "3913a2c9-e88c-4f49-bc63-02b1fa543c38" (UID: "3913a2c9-e88c-4f49-bc63-02b1fa543c38"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:08:56.228992 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.228969 5949 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-k8s-db\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:08:56.228992 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.228993 5949 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:08:56.229126 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.229003 5949 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-thanos-prometheus-http-client-file\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:08:56.229126 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.229013 5949 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-prometheus-k8s-tls\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:08:56.229126 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.229023 5949 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:08:56.229126 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.229032 5949 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-kube-rbac-proxy\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:08:56.229126 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.229050 5949 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-web-config\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:08:56.229126 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.229058 5949 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-configmap-metrics-client-ca\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:08:56.229126 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.229067 5949 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3913a2c9-e88c-4f49-bc63-02b1fa543c38-tls-assets\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:08:56.229126 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.229075 5949 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qsvp8\" (UniqueName: \"kubernetes.io/projected/3913a2c9-e88c-4f49-bc63-02b1fa543c38-kube-api-access-qsvp8\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:08:56.229126 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.229083 5949 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-config\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:08:56.229126 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.229090 5949 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-grpc-tls\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:08:56.229126 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.229099 5949 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:08:56.229126 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.229108 5949 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:08:56.229126 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.229116 5949 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3913a2c9-e88c-4f49-bc63-02b1fa543c38-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:08:56.229126 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.229125 5949 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3913a2c9-e88c-4f49-bc63-02b1fa543c38-secret-metrics-client-certs\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:08:56.229523 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.229134 5949 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3913a2c9-e88c-4f49-bc63-02b1fa543c38-config-out\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:08:56.338545 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.338513 5949 generic.go:356] "Generic (PLEG): container finished" podID="3913a2c9-e88c-4f49-bc63-02b1fa543c38" containerID="b2e637f872238fab5e1353aa253e2d1ec3b076550b69a82adb4d137d1af394d4" exitCode=0 Oct 30 22:08:56.338545 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.338540 5949 generic.go:356] "Generic (PLEG): container finished" podID="3913a2c9-e88c-4f49-bc63-02b1fa543c38" containerID="e65b0800081ec131f68d9c0efae55c42ac26fdd495967642fa18f904d0840737" exitCode=0 Oct 30 22:08:56.338545 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.338546 5949 generic.go:356] "Generic (PLEG): container finished" podID="3913a2c9-e88c-4f49-bc63-02b1fa543c38" containerID="911a2c5672e542f3af43a336f66776e42858c381f94b8a1fe92b1ca4b805c5f5" exitCode=0 Oct 30 22:08:56.338998 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.338552 5949 generic.go:356] "Generic (PLEG): container finished" podID="3913a2c9-e88c-4f49-bc63-02b1fa543c38" containerID="056a499f0455153c6c6409a71b6292bf1c957cd5d23143a835b0e63819b58859" exitCode=0 Oct 30 22:08:56.338998 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.338560 5949 generic.go:356] "Generic (PLEG): container finished" podID="3913a2c9-e88c-4f49-bc63-02b1fa543c38" containerID="6df39d2515c9a702cf3a1d64f1cc0a8a9e056bbb0fa98cf0ebaad9dad3096a05" exitCode=0 Oct 30 22:08:56.338998 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.338568 5949 generic.go:356] "Generic (PLEG): container finished" podID="3913a2c9-e88c-4f49-bc63-02b1fa543c38" containerID="36bd1a5bed3b82469994cb7d98f25114091efa66af5fe091e3037500ac0d995b" exitCode=0 Oct 30 22:08:56.338998 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.338592 5949 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.338998 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.338589 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3913a2c9-e88c-4f49-bc63-02b1fa543c38","Type":"ContainerDied","Data":"b2e637f872238fab5e1353aa253e2d1ec3b076550b69a82adb4d137d1af394d4"} Oct 30 22:08:56.338998 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.338658 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3913a2c9-e88c-4f49-bc63-02b1fa543c38","Type":"ContainerDied","Data":"e65b0800081ec131f68d9c0efae55c42ac26fdd495967642fa18f904d0840737"} Oct 30 22:08:56.338998 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.338686 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3913a2c9-e88c-4f49-bc63-02b1fa543c38","Type":"ContainerDied","Data":"911a2c5672e542f3af43a336f66776e42858c381f94b8a1fe92b1ca4b805c5f5"} Oct 30 22:08:56.338998 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.338711 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3913a2c9-e88c-4f49-bc63-02b1fa543c38","Type":"ContainerDied","Data":"056a499f0455153c6c6409a71b6292bf1c957cd5d23143a835b0e63819b58859"} Oct 30 22:08:56.338998 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.338726 5949 scope.go:122] "RemoveContainer" containerID="b2e637f872238fab5e1353aa253e2d1ec3b076550b69a82adb4d137d1af394d4" Oct 30 22:08:56.338998 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.338736 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3913a2c9-e88c-4f49-bc63-02b1fa543c38","Type":"ContainerDied","Data":"6df39d2515c9a702cf3a1d64f1cc0a8a9e056bbb0fa98cf0ebaad9dad3096a05"} Oct 30 22:08:56.338998 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.338761 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3913a2c9-e88c-4f49-bc63-02b1fa543c38","Type":"ContainerDied","Data":"36bd1a5bed3b82469994cb7d98f25114091efa66af5fe091e3037500ac0d995b"} Oct 30 22:08:56.338998 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.338785 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3913a2c9-e88c-4f49-bc63-02b1fa543c38","Type":"ContainerDied","Data":"2e1024a48494975a37cd70c11675061811824847d6e8d357b5b6836a0f397c37"} Oct 30 22:08:56.346364 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.346345 5949 scope.go:122] "RemoveContainer" containerID="e65b0800081ec131f68d9c0efae55c42ac26fdd495967642fa18f904d0840737" Oct 30 22:08:56.353162 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.353143 5949 scope.go:122] "RemoveContainer" containerID="911a2c5672e542f3af43a336f66776e42858c381f94b8a1fe92b1ca4b805c5f5" Oct 30 22:08:56.359721 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.359704 5949 scope.go:122] "RemoveContainer" containerID="056a499f0455153c6c6409a71b6292bf1c957cd5d23143a835b0e63819b58859" Oct 30 22:08:56.366180 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.366161 5949 scope.go:122] "RemoveContainer" containerID="6df39d2515c9a702cf3a1d64f1cc0a8a9e056bbb0fa98cf0ebaad9dad3096a05" Oct 30 22:08:56.374969 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.374945 5949 scope.go:122] "RemoveContainer" containerID="36bd1a5bed3b82469994cb7d98f25114091efa66af5fe091e3037500ac0d995b" Oct 30 22:08:56.382286 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.382261 5949 scope.go:122] "RemoveContainer" containerID="8b9b1cb2d4b58b1640e7b7ffa96201c957c0f209ed4cbb0101202683c3db5f62" Oct 30 22:08:56.388659 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.388643 5949 scope.go:122] "RemoveContainer" containerID="b2e637f872238fab5e1353aa253e2d1ec3b076550b69a82adb4d137d1af394d4" Oct 30 22:08:56.388960 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:56.388939 5949 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2e637f872238fab5e1353aa253e2d1ec3b076550b69a82adb4d137d1af394d4\": container with ID starting with b2e637f872238fab5e1353aa253e2d1ec3b076550b69a82adb4d137d1af394d4 not found: ID does not exist" containerID="b2e637f872238fab5e1353aa253e2d1ec3b076550b69a82adb4d137d1af394d4" Oct 30 22:08:56.389020 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.388968 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2e637f872238fab5e1353aa253e2d1ec3b076550b69a82adb4d137d1af394d4"} err="failed to get container status \"b2e637f872238fab5e1353aa253e2d1ec3b076550b69a82adb4d137d1af394d4\": rpc error: code = NotFound desc = could not find container \"b2e637f872238fab5e1353aa253e2d1ec3b076550b69a82adb4d137d1af394d4\": container with ID starting with b2e637f872238fab5e1353aa253e2d1ec3b076550b69a82adb4d137d1af394d4 not found: ID does not exist" Oct 30 22:08:56.389020 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.388997 5949 scope.go:122] "RemoveContainer" containerID="e65b0800081ec131f68d9c0efae55c42ac26fdd495967642fa18f904d0840737" Oct 30 22:08:56.389270 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:56.389211 5949 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e65b0800081ec131f68d9c0efae55c42ac26fdd495967642fa18f904d0840737\": container with ID starting with e65b0800081ec131f68d9c0efae55c42ac26fdd495967642fa18f904d0840737 not found: ID does not exist" containerID="e65b0800081ec131f68d9c0efae55c42ac26fdd495967642fa18f904d0840737" Oct 30 22:08:56.389363 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.389271 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e65b0800081ec131f68d9c0efae55c42ac26fdd495967642fa18f904d0840737"} err="failed to get container status \"e65b0800081ec131f68d9c0efae55c42ac26fdd495967642fa18f904d0840737\": rpc error: code = NotFound desc = could not find container \"e65b0800081ec131f68d9c0efae55c42ac26fdd495967642fa18f904d0840737\": container with ID starting with e65b0800081ec131f68d9c0efae55c42ac26fdd495967642fa18f904d0840737 not found: ID does not exist" Oct 30 22:08:56.389363 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.389284 5949 scope.go:122] "RemoveContainer" containerID="911a2c5672e542f3af43a336f66776e42858c381f94b8a1fe92b1ca4b805c5f5" Oct 30 22:08:56.389528 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:56.389512 5949 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"911a2c5672e542f3af43a336f66776e42858c381f94b8a1fe92b1ca4b805c5f5\": container with ID starting with 911a2c5672e542f3af43a336f66776e42858c381f94b8a1fe92b1ca4b805c5f5 not found: ID does not exist" containerID="911a2c5672e542f3af43a336f66776e42858c381f94b8a1fe92b1ca4b805c5f5" Oct 30 22:08:56.389569 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.389531 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"911a2c5672e542f3af43a336f66776e42858c381f94b8a1fe92b1ca4b805c5f5"} err="failed to get container status \"911a2c5672e542f3af43a336f66776e42858c381f94b8a1fe92b1ca4b805c5f5\": rpc error: code = NotFound desc = could not find container \"911a2c5672e542f3af43a336f66776e42858c381f94b8a1fe92b1ca4b805c5f5\": container with ID starting with 911a2c5672e542f3af43a336f66776e42858c381f94b8a1fe92b1ca4b805c5f5 not found: ID does not exist" Oct 30 22:08:56.389569 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.389543 5949 scope.go:122] "RemoveContainer" containerID="056a499f0455153c6c6409a71b6292bf1c957cd5d23143a835b0e63819b58859" Oct 30 22:08:56.389765 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:56.389746 5949 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"056a499f0455153c6c6409a71b6292bf1c957cd5d23143a835b0e63819b58859\": container with ID starting with 056a499f0455153c6c6409a71b6292bf1c957cd5d23143a835b0e63819b58859 not found: ID does not exist" containerID="056a499f0455153c6c6409a71b6292bf1c957cd5d23143a835b0e63819b58859" Oct 30 22:08:56.389831 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.389774 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"056a499f0455153c6c6409a71b6292bf1c957cd5d23143a835b0e63819b58859"} err="failed to get container status \"056a499f0455153c6c6409a71b6292bf1c957cd5d23143a835b0e63819b58859\": rpc error: code = NotFound desc = could not find container \"056a499f0455153c6c6409a71b6292bf1c957cd5d23143a835b0e63819b58859\": container with ID starting with 056a499f0455153c6c6409a71b6292bf1c957cd5d23143a835b0e63819b58859 not found: ID does not exist" Oct 30 22:08:56.389831 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.389795 5949 scope.go:122] "RemoveContainer" containerID="6df39d2515c9a702cf3a1d64f1cc0a8a9e056bbb0fa98cf0ebaad9dad3096a05" Oct 30 22:08:56.390071 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:56.390051 5949 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6df39d2515c9a702cf3a1d64f1cc0a8a9e056bbb0fa98cf0ebaad9dad3096a05\": container with ID starting with 6df39d2515c9a702cf3a1d64f1cc0a8a9e056bbb0fa98cf0ebaad9dad3096a05 not found: ID does not exist" containerID="6df39d2515c9a702cf3a1d64f1cc0a8a9e056bbb0fa98cf0ebaad9dad3096a05" Oct 30 22:08:56.390146 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.390074 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6df39d2515c9a702cf3a1d64f1cc0a8a9e056bbb0fa98cf0ebaad9dad3096a05"} err="failed to get container status \"6df39d2515c9a702cf3a1d64f1cc0a8a9e056bbb0fa98cf0ebaad9dad3096a05\": rpc error: code = NotFound desc = could not find container \"6df39d2515c9a702cf3a1d64f1cc0a8a9e056bbb0fa98cf0ebaad9dad3096a05\": container with ID starting with 6df39d2515c9a702cf3a1d64f1cc0a8a9e056bbb0fa98cf0ebaad9dad3096a05 not found: ID does not exist" Oct 30 22:08:56.390146 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.390087 5949 scope.go:122] "RemoveContainer" containerID="36bd1a5bed3b82469994cb7d98f25114091efa66af5fe091e3037500ac0d995b" Oct 30 22:08:56.390305 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:56.390289 5949 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36bd1a5bed3b82469994cb7d98f25114091efa66af5fe091e3037500ac0d995b\": container with ID starting with 36bd1a5bed3b82469994cb7d98f25114091efa66af5fe091e3037500ac0d995b not found: ID does not exist" containerID="36bd1a5bed3b82469994cb7d98f25114091efa66af5fe091e3037500ac0d995b" Oct 30 22:08:56.390347 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.390308 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36bd1a5bed3b82469994cb7d98f25114091efa66af5fe091e3037500ac0d995b"} err="failed to get container status \"36bd1a5bed3b82469994cb7d98f25114091efa66af5fe091e3037500ac0d995b\": rpc error: code = NotFound desc = could not find container \"36bd1a5bed3b82469994cb7d98f25114091efa66af5fe091e3037500ac0d995b\": container with ID starting with 36bd1a5bed3b82469994cb7d98f25114091efa66af5fe091e3037500ac0d995b not found: ID does not exist" Oct 30 22:08:56.390347 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.390325 5949 scope.go:122] "RemoveContainer" containerID="8b9b1cb2d4b58b1640e7b7ffa96201c957c0f209ed4cbb0101202683c3db5f62" Oct 30 22:08:56.390540 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:08:56.390517 5949 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b9b1cb2d4b58b1640e7b7ffa96201c957c0f209ed4cbb0101202683c3db5f62\": container with ID starting with 8b9b1cb2d4b58b1640e7b7ffa96201c957c0f209ed4cbb0101202683c3db5f62 not found: ID does not exist" containerID="8b9b1cb2d4b58b1640e7b7ffa96201c957c0f209ed4cbb0101202683c3db5f62" Oct 30 22:08:56.390588 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.390536 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b9b1cb2d4b58b1640e7b7ffa96201c957c0f209ed4cbb0101202683c3db5f62"} err="failed to get container status \"8b9b1cb2d4b58b1640e7b7ffa96201c957c0f209ed4cbb0101202683c3db5f62\": rpc error: code = NotFound desc = could not find container \"8b9b1cb2d4b58b1640e7b7ffa96201c957c0f209ed4cbb0101202683c3db5f62\": container with ID starting with 8b9b1cb2d4b58b1640e7b7ffa96201c957c0f209ed4cbb0101202683c3db5f62 not found: ID does not exist" Oct 30 22:08:56.390588 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.390548 5949 scope.go:122] "RemoveContainer" containerID="b2e637f872238fab5e1353aa253e2d1ec3b076550b69a82adb4d137d1af394d4" Oct 30 22:08:56.390760 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.390737 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2e637f872238fab5e1353aa253e2d1ec3b076550b69a82adb4d137d1af394d4"} err="failed to get container status \"b2e637f872238fab5e1353aa253e2d1ec3b076550b69a82adb4d137d1af394d4\": rpc error: code = NotFound desc = could not find container \"b2e637f872238fab5e1353aa253e2d1ec3b076550b69a82adb4d137d1af394d4\": container with ID starting with b2e637f872238fab5e1353aa253e2d1ec3b076550b69a82adb4d137d1af394d4 not found: ID does not exist" Oct 30 22:08:56.390760 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.390752 5949 scope.go:122] "RemoveContainer" containerID="e65b0800081ec131f68d9c0efae55c42ac26fdd495967642fa18f904d0840737" Oct 30 22:08:56.390949 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.390933 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e65b0800081ec131f68d9c0efae55c42ac26fdd495967642fa18f904d0840737"} err="failed to get container status \"e65b0800081ec131f68d9c0efae55c42ac26fdd495967642fa18f904d0840737\": rpc error: code = NotFound desc = could not find container \"e65b0800081ec131f68d9c0efae55c42ac26fdd495967642fa18f904d0840737\": container with ID starting with e65b0800081ec131f68d9c0efae55c42ac26fdd495967642fa18f904d0840737 not found: ID does not exist" Oct 30 22:08:56.390990 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.390948 5949 scope.go:122] "RemoveContainer" containerID="911a2c5672e542f3af43a336f66776e42858c381f94b8a1fe92b1ca4b805c5f5" Oct 30 22:08:56.391134 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.391118 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"911a2c5672e542f3af43a336f66776e42858c381f94b8a1fe92b1ca4b805c5f5"} err="failed to get container status \"911a2c5672e542f3af43a336f66776e42858c381f94b8a1fe92b1ca4b805c5f5\": rpc error: code = NotFound desc = could not find container \"911a2c5672e542f3af43a336f66776e42858c381f94b8a1fe92b1ca4b805c5f5\": container with ID starting with 911a2c5672e542f3af43a336f66776e42858c381f94b8a1fe92b1ca4b805c5f5 not found: ID does not exist" Oct 30 22:08:56.391177 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.391133 5949 scope.go:122] "RemoveContainer" containerID="056a499f0455153c6c6409a71b6292bf1c957cd5d23143a835b0e63819b58859" Oct 30 22:08:56.391358 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.391342 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"056a499f0455153c6c6409a71b6292bf1c957cd5d23143a835b0e63819b58859"} err="failed to get container status \"056a499f0455153c6c6409a71b6292bf1c957cd5d23143a835b0e63819b58859\": rpc error: code = NotFound desc = could not find container \"056a499f0455153c6c6409a71b6292bf1c957cd5d23143a835b0e63819b58859\": container with ID starting with 056a499f0455153c6c6409a71b6292bf1c957cd5d23143a835b0e63819b58859 not found: ID does not exist" Oct 30 22:08:56.391412 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.391358 5949 scope.go:122] "RemoveContainer" containerID="6df39d2515c9a702cf3a1d64f1cc0a8a9e056bbb0fa98cf0ebaad9dad3096a05" Oct 30 22:08:56.391556 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.391541 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6df39d2515c9a702cf3a1d64f1cc0a8a9e056bbb0fa98cf0ebaad9dad3096a05"} err="failed to get container status \"6df39d2515c9a702cf3a1d64f1cc0a8a9e056bbb0fa98cf0ebaad9dad3096a05\": rpc error: code = NotFound desc = could not find container \"6df39d2515c9a702cf3a1d64f1cc0a8a9e056bbb0fa98cf0ebaad9dad3096a05\": container with ID starting with 6df39d2515c9a702cf3a1d64f1cc0a8a9e056bbb0fa98cf0ebaad9dad3096a05 not found: ID does not exist" Oct 30 22:08:56.391597 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.391555 5949 scope.go:122] "RemoveContainer" containerID="36bd1a5bed3b82469994cb7d98f25114091efa66af5fe091e3037500ac0d995b" Oct 30 22:08:56.391755 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.391741 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36bd1a5bed3b82469994cb7d98f25114091efa66af5fe091e3037500ac0d995b"} err="failed to get container status \"36bd1a5bed3b82469994cb7d98f25114091efa66af5fe091e3037500ac0d995b\": rpc error: code = NotFound desc = could not find container \"36bd1a5bed3b82469994cb7d98f25114091efa66af5fe091e3037500ac0d995b\": container with ID starting with 36bd1a5bed3b82469994cb7d98f25114091efa66af5fe091e3037500ac0d995b not found: ID does not exist" Oct 30 22:08:56.391798 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.391755 5949 scope.go:122] "RemoveContainer" containerID="8b9b1cb2d4b58b1640e7b7ffa96201c957c0f209ed4cbb0101202683c3db5f62" Oct 30 22:08:56.391963 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.391947 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b9b1cb2d4b58b1640e7b7ffa96201c957c0f209ed4cbb0101202683c3db5f62"} err="failed to get container status \"8b9b1cb2d4b58b1640e7b7ffa96201c957c0f209ed4cbb0101202683c3db5f62\": rpc error: code = NotFound desc = could not find container \"8b9b1cb2d4b58b1640e7b7ffa96201c957c0f209ed4cbb0101202683c3db5f62\": container with ID starting with 8b9b1cb2d4b58b1640e7b7ffa96201c957c0f209ed4cbb0101202683c3db5f62 not found: ID does not exist" Oct 30 22:08:56.391963 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.391962 5949 scope.go:122] "RemoveContainer" containerID="b2e637f872238fab5e1353aa253e2d1ec3b076550b69a82adb4d137d1af394d4" Oct 30 22:08:56.392150 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.392136 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2e637f872238fab5e1353aa253e2d1ec3b076550b69a82adb4d137d1af394d4"} err="failed to get container status \"b2e637f872238fab5e1353aa253e2d1ec3b076550b69a82adb4d137d1af394d4\": rpc error: code = NotFound desc = could not find container \"b2e637f872238fab5e1353aa253e2d1ec3b076550b69a82adb4d137d1af394d4\": container with ID starting with b2e637f872238fab5e1353aa253e2d1ec3b076550b69a82adb4d137d1af394d4 not found: ID does not exist" Oct 30 22:08:56.392194 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.392150 5949 scope.go:122] "RemoveContainer" containerID="e65b0800081ec131f68d9c0efae55c42ac26fdd495967642fa18f904d0840737" Oct 30 22:08:56.392350 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.392336 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e65b0800081ec131f68d9c0efae55c42ac26fdd495967642fa18f904d0840737"} err="failed to get container status \"e65b0800081ec131f68d9c0efae55c42ac26fdd495967642fa18f904d0840737\": rpc error: code = NotFound desc = could not find container \"e65b0800081ec131f68d9c0efae55c42ac26fdd495967642fa18f904d0840737\": container with ID starting with e65b0800081ec131f68d9c0efae55c42ac26fdd495967642fa18f904d0840737 not found: ID does not exist" Oct 30 22:08:56.392394 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.392351 5949 scope.go:122] "RemoveContainer" containerID="911a2c5672e542f3af43a336f66776e42858c381f94b8a1fe92b1ca4b805c5f5" Oct 30 22:08:56.392546 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.392531 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"911a2c5672e542f3af43a336f66776e42858c381f94b8a1fe92b1ca4b805c5f5"} err="failed to get container status \"911a2c5672e542f3af43a336f66776e42858c381f94b8a1fe92b1ca4b805c5f5\": rpc error: code = NotFound desc = could not find container \"911a2c5672e542f3af43a336f66776e42858c381f94b8a1fe92b1ca4b805c5f5\": container with ID starting with 911a2c5672e542f3af43a336f66776e42858c381f94b8a1fe92b1ca4b805c5f5 not found: ID does not exist" Oct 30 22:08:56.392546 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.392546 5949 scope.go:122] "RemoveContainer" containerID="056a499f0455153c6c6409a71b6292bf1c957cd5d23143a835b0e63819b58859" Oct 30 22:08:56.392735 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.392719 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"056a499f0455153c6c6409a71b6292bf1c957cd5d23143a835b0e63819b58859"} err="failed to get container status \"056a499f0455153c6c6409a71b6292bf1c957cd5d23143a835b0e63819b58859\": rpc error: code = NotFound desc = could not find container \"056a499f0455153c6c6409a71b6292bf1c957cd5d23143a835b0e63819b58859\": container with ID starting with 056a499f0455153c6c6409a71b6292bf1c957cd5d23143a835b0e63819b58859 not found: ID does not exist" Oct 30 22:08:56.392735 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.392733 5949 scope.go:122] "RemoveContainer" containerID="6df39d2515c9a702cf3a1d64f1cc0a8a9e056bbb0fa98cf0ebaad9dad3096a05" Oct 30 22:08:56.392925 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.392909 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6df39d2515c9a702cf3a1d64f1cc0a8a9e056bbb0fa98cf0ebaad9dad3096a05"} err="failed to get container status \"6df39d2515c9a702cf3a1d64f1cc0a8a9e056bbb0fa98cf0ebaad9dad3096a05\": rpc error: code = NotFound desc = could not find container \"6df39d2515c9a702cf3a1d64f1cc0a8a9e056bbb0fa98cf0ebaad9dad3096a05\": container with ID starting with 6df39d2515c9a702cf3a1d64f1cc0a8a9e056bbb0fa98cf0ebaad9dad3096a05 not found: ID does not exist" Oct 30 22:08:56.392925 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.392923 5949 scope.go:122] "RemoveContainer" containerID="36bd1a5bed3b82469994cb7d98f25114091efa66af5fe091e3037500ac0d995b" Oct 30 22:08:56.393111 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.393096 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36bd1a5bed3b82469994cb7d98f25114091efa66af5fe091e3037500ac0d995b"} err="failed to get container status \"36bd1a5bed3b82469994cb7d98f25114091efa66af5fe091e3037500ac0d995b\": rpc error: code = NotFound desc = could not find container \"36bd1a5bed3b82469994cb7d98f25114091efa66af5fe091e3037500ac0d995b\": container with ID starting with 36bd1a5bed3b82469994cb7d98f25114091efa66af5fe091e3037500ac0d995b not found: ID does not exist" Oct 30 22:08:56.393158 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.393111 5949 scope.go:122] "RemoveContainer" containerID="8b9b1cb2d4b58b1640e7b7ffa96201c957c0f209ed4cbb0101202683c3db5f62" Oct 30 22:08:56.393321 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.393307 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b9b1cb2d4b58b1640e7b7ffa96201c957c0f209ed4cbb0101202683c3db5f62"} err="failed to get container status \"8b9b1cb2d4b58b1640e7b7ffa96201c957c0f209ed4cbb0101202683c3db5f62\": rpc error: code = NotFound desc = could not find container \"8b9b1cb2d4b58b1640e7b7ffa96201c957c0f209ed4cbb0101202683c3db5f62\": container with ID starting with 8b9b1cb2d4b58b1640e7b7ffa96201c957c0f209ed4cbb0101202683c3db5f62 not found: ID does not exist" Oct 30 22:08:56.393365 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.393321 5949 scope.go:122] "RemoveContainer" containerID="b2e637f872238fab5e1353aa253e2d1ec3b076550b69a82adb4d137d1af394d4" Oct 30 22:08:56.393507 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.393493 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2e637f872238fab5e1353aa253e2d1ec3b076550b69a82adb4d137d1af394d4"} err="failed to get container status \"b2e637f872238fab5e1353aa253e2d1ec3b076550b69a82adb4d137d1af394d4\": rpc error: code = NotFound desc = could not find container \"b2e637f872238fab5e1353aa253e2d1ec3b076550b69a82adb4d137d1af394d4\": container with ID starting with b2e637f872238fab5e1353aa253e2d1ec3b076550b69a82adb4d137d1af394d4 not found: ID does not exist" Oct 30 22:08:56.393549 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.393507 5949 scope.go:122] "RemoveContainer" containerID="e65b0800081ec131f68d9c0efae55c42ac26fdd495967642fa18f904d0840737" Oct 30 22:08:56.393715 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.393695 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e65b0800081ec131f68d9c0efae55c42ac26fdd495967642fa18f904d0840737"} err="failed to get container status \"e65b0800081ec131f68d9c0efae55c42ac26fdd495967642fa18f904d0840737\": rpc error: code = NotFound desc = could not find container \"e65b0800081ec131f68d9c0efae55c42ac26fdd495967642fa18f904d0840737\": container with ID starting with e65b0800081ec131f68d9c0efae55c42ac26fdd495967642fa18f904d0840737 not found: ID does not exist" Oct 30 22:08:56.393715 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.393714 5949 scope.go:122] "RemoveContainer" containerID="911a2c5672e542f3af43a336f66776e42858c381f94b8a1fe92b1ca4b805c5f5" Oct 30 22:08:56.393970 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.393950 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"911a2c5672e542f3af43a336f66776e42858c381f94b8a1fe92b1ca4b805c5f5"} err="failed to get container status \"911a2c5672e542f3af43a336f66776e42858c381f94b8a1fe92b1ca4b805c5f5\": rpc error: code = NotFound desc = could not find container \"911a2c5672e542f3af43a336f66776e42858c381f94b8a1fe92b1ca4b805c5f5\": container with ID starting with 911a2c5672e542f3af43a336f66776e42858c381f94b8a1fe92b1ca4b805c5f5 not found: ID does not exist" Oct 30 22:08:56.394037 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.393979 5949 scope.go:122] "RemoveContainer" containerID="056a499f0455153c6c6409a71b6292bf1c957cd5d23143a835b0e63819b58859" Oct 30 22:08:56.394278 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.394252 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"056a499f0455153c6c6409a71b6292bf1c957cd5d23143a835b0e63819b58859"} err="failed to get container status \"056a499f0455153c6c6409a71b6292bf1c957cd5d23143a835b0e63819b58859\": rpc error: code = NotFound desc = could not find container \"056a499f0455153c6c6409a71b6292bf1c957cd5d23143a835b0e63819b58859\": container with ID starting with 056a499f0455153c6c6409a71b6292bf1c957cd5d23143a835b0e63819b58859 not found: ID does not exist" Oct 30 22:08:56.394278 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.394277 5949 scope.go:122] "RemoveContainer" containerID="6df39d2515c9a702cf3a1d64f1cc0a8a9e056bbb0fa98cf0ebaad9dad3096a05" Oct 30 22:08:56.394562 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.394539 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6df39d2515c9a702cf3a1d64f1cc0a8a9e056bbb0fa98cf0ebaad9dad3096a05"} err="failed to get container status \"6df39d2515c9a702cf3a1d64f1cc0a8a9e056bbb0fa98cf0ebaad9dad3096a05\": rpc error: code = NotFound desc = could not find container \"6df39d2515c9a702cf3a1d64f1cc0a8a9e056bbb0fa98cf0ebaad9dad3096a05\": container with ID starting with 6df39d2515c9a702cf3a1d64f1cc0a8a9e056bbb0fa98cf0ebaad9dad3096a05 not found: ID does not exist" Oct 30 22:08:56.394606 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.394564 5949 scope.go:122] "RemoveContainer" containerID="36bd1a5bed3b82469994cb7d98f25114091efa66af5fe091e3037500ac0d995b" Oct 30 22:08:56.394815 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.394797 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36bd1a5bed3b82469994cb7d98f25114091efa66af5fe091e3037500ac0d995b"} err="failed to get container status \"36bd1a5bed3b82469994cb7d98f25114091efa66af5fe091e3037500ac0d995b\": rpc error: code = NotFound desc = could not find container \"36bd1a5bed3b82469994cb7d98f25114091efa66af5fe091e3037500ac0d995b\": container with ID starting with 36bd1a5bed3b82469994cb7d98f25114091efa66af5fe091e3037500ac0d995b not found: ID does not exist" Oct 30 22:08:56.394854 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.394816 5949 scope.go:122] "RemoveContainer" containerID="8b9b1cb2d4b58b1640e7b7ffa96201c957c0f209ed4cbb0101202683c3db5f62" Oct 30 22:08:56.394891 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.394882 5949 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 30 22:08:56.395037 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.395017 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b9b1cb2d4b58b1640e7b7ffa96201c957c0f209ed4cbb0101202683c3db5f62"} err="failed to get container status \"8b9b1cb2d4b58b1640e7b7ffa96201c957c0f209ed4cbb0101202683c3db5f62\": rpc error: code = NotFound desc = could not find container \"8b9b1cb2d4b58b1640e7b7ffa96201c957c0f209ed4cbb0101202683c3db5f62\": container with ID starting with 8b9b1cb2d4b58b1640e7b7ffa96201c957c0f209ed4cbb0101202683c3db5f62 not found: ID does not exist" Oct 30 22:08:56.395075 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.395038 5949 scope.go:122] "RemoveContainer" containerID="b2e637f872238fab5e1353aa253e2d1ec3b076550b69a82adb4d137d1af394d4" Oct 30 22:08:56.395276 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.395255 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2e637f872238fab5e1353aa253e2d1ec3b076550b69a82adb4d137d1af394d4"} err="failed to get container status \"b2e637f872238fab5e1353aa253e2d1ec3b076550b69a82adb4d137d1af394d4\": rpc error: code = NotFound desc = could not find container \"b2e637f872238fab5e1353aa253e2d1ec3b076550b69a82adb4d137d1af394d4\": container with ID starting with b2e637f872238fab5e1353aa253e2d1ec3b076550b69a82adb4d137d1af394d4 not found: ID does not exist" Oct 30 22:08:56.395276 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.395274 5949 scope.go:122] "RemoveContainer" containerID="e65b0800081ec131f68d9c0efae55c42ac26fdd495967642fa18f904d0840737" Oct 30 22:08:56.395545 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.395525 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e65b0800081ec131f68d9c0efae55c42ac26fdd495967642fa18f904d0840737"} err="failed to get container status \"e65b0800081ec131f68d9c0efae55c42ac26fdd495967642fa18f904d0840737\": rpc error: code = NotFound desc = could not find container \"e65b0800081ec131f68d9c0efae55c42ac26fdd495967642fa18f904d0840737\": container with ID starting with e65b0800081ec131f68d9c0efae55c42ac26fdd495967642fa18f904d0840737 not found: ID does not exist" Oct 30 22:08:56.395545 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.395544 5949 scope.go:122] "RemoveContainer" containerID="911a2c5672e542f3af43a336f66776e42858c381f94b8a1fe92b1ca4b805c5f5" Oct 30 22:08:56.395882 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.395855 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"911a2c5672e542f3af43a336f66776e42858c381f94b8a1fe92b1ca4b805c5f5"} err="failed to get container status \"911a2c5672e542f3af43a336f66776e42858c381f94b8a1fe92b1ca4b805c5f5\": rpc error: code = NotFound desc = could not find container \"911a2c5672e542f3af43a336f66776e42858c381f94b8a1fe92b1ca4b805c5f5\": container with ID starting with 911a2c5672e542f3af43a336f66776e42858c381f94b8a1fe92b1ca4b805c5f5 not found: ID does not exist" Oct 30 22:08:56.395977 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.395883 5949 scope.go:122] "RemoveContainer" containerID="056a499f0455153c6c6409a71b6292bf1c957cd5d23143a835b0e63819b58859" Oct 30 22:08:56.396210 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.396185 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"056a499f0455153c6c6409a71b6292bf1c957cd5d23143a835b0e63819b58859"} err="failed to get container status \"056a499f0455153c6c6409a71b6292bf1c957cd5d23143a835b0e63819b58859\": rpc error: code = NotFound desc = could not find container \"056a499f0455153c6c6409a71b6292bf1c957cd5d23143a835b0e63819b58859\": container with ID starting with 056a499f0455153c6c6409a71b6292bf1c957cd5d23143a835b0e63819b58859 not found: ID does not exist" Oct 30 22:08:56.396294 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.396210 5949 scope.go:122] "RemoveContainer" containerID="6df39d2515c9a702cf3a1d64f1cc0a8a9e056bbb0fa98cf0ebaad9dad3096a05" Oct 30 22:08:56.396548 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.396506 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6df39d2515c9a702cf3a1d64f1cc0a8a9e056bbb0fa98cf0ebaad9dad3096a05"} err="failed to get container status \"6df39d2515c9a702cf3a1d64f1cc0a8a9e056bbb0fa98cf0ebaad9dad3096a05\": rpc error: code = NotFound desc = could not find container \"6df39d2515c9a702cf3a1d64f1cc0a8a9e056bbb0fa98cf0ebaad9dad3096a05\": container with ID starting with 6df39d2515c9a702cf3a1d64f1cc0a8a9e056bbb0fa98cf0ebaad9dad3096a05 not found: ID does not exist" Oct 30 22:08:56.396548 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.396531 5949 scope.go:122] "RemoveContainer" containerID="36bd1a5bed3b82469994cb7d98f25114091efa66af5fe091e3037500ac0d995b" Oct 30 22:08:56.396799 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.396782 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36bd1a5bed3b82469994cb7d98f25114091efa66af5fe091e3037500ac0d995b"} err="failed to get container status \"36bd1a5bed3b82469994cb7d98f25114091efa66af5fe091e3037500ac0d995b\": rpc error: code = NotFound desc = could not find container \"36bd1a5bed3b82469994cb7d98f25114091efa66af5fe091e3037500ac0d995b\": container with ID starting with 36bd1a5bed3b82469994cb7d98f25114091efa66af5fe091e3037500ac0d995b not found: ID does not exist" Oct 30 22:08:56.396847 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.396808 5949 scope.go:122] "RemoveContainer" containerID="8b9b1cb2d4b58b1640e7b7ffa96201c957c0f209ed4cbb0101202683c3db5f62" Oct 30 22:08:56.397034 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.397017 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b9b1cb2d4b58b1640e7b7ffa96201c957c0f209ed4cbb0101202683c3db5f62"} err="failed to get container status \"8b9b1cb2d4b58b1640e7b7ffa96201c957c0f209ed4cbb0101202683c3db5f62\": rpc error: code = NotFound desc = could not find container \"8b9b1cb2d4b58b1640e7b7ffa96201c957c0f209ed4cbb0101202683c3db5f62\": container with ID starting with 8b9b1cb2d4b58b1640e7b7ffa96201c957c0f209ed4cbb0101202683c3db5f62 not found: ID does not exist" Oct 30 22:08:56.397088 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.397034 5949 scope.go:122] "RemoveContainer" containerID="b2e637f872238fab5e1353aa253e2d1ec3b076550b69a82adb4d137d1af394d4" Oct 30 22:08:56.397281 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.397252 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2e637f872238fab5e1353aa253e2d1ec3b076550b69a82adb4d137d1af394d4"} err="failed to get container status \"b2e637f872238fab5e1353aa253e2d1ec3b076550b69a82adb4d137d1af394d4\": rpc error: code = NotFound desc = could not find container \"b2e637f872238fab5e1353aa253e2d1ec3b076550b69a82adb4d137d1af394d4\": container with ID starting with b2e637f872238fab5e1353aa253e2d1ec3b076550b69a82adb4d137d1af394d4 not found: ID does not exist" Oct 30 22:08:56.397336 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.397281 5949 scope.go:122] "RemoveContainer" containerID="e65b0800081ec131f68d9c0efae55c42ac26fdd495967642fa18f904d0840737" Oct 30 22:08:56.397513 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.397498 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e65b0800081ec131f68d9c0efae55c42ac26fdd495967642fa18f904d0840737"} err="failed to get container status \"e65b0800081ec131f68d9c0efae55c42ac26fdd495967642fa18f904d0840737\": rpc error: code = NotFound desc = could not find container \"e65b0800081ec131f68d9c0efae55c42ac26fdd495967642fa18f904d0840737\": container with ID starting with e65b0800081ec131f68d9c0efae55c42ac26fdd495967642fa18f904d0840737 not found: ID does not exist" Oct 30 22:08:56.397513 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.397514 5949 scope.go:122] "RemoveContainer" containerID="911a2c5672e542f3af43a336f66776e42858c381f94b8a1fe92b1ca4b805c5f5" Oct 30 22:08:56.397717 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.397697 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"911a2c5672e542f3af43a336f66776e42858c381f94b8a1fe92b1ca4b805c5f5"} err="failed to get container status \"911a2c5672e542f3af43a336f66776e42858c381f94b8a1fe92b1ca4b805c5f5\": rpc error: code = NotFound desc = could not find container \"911a2c5672e542f3af43a336f66776e42858c381f94b8a1fe92b1ca4b805c5f5\": container with ID starting with 911a2c5672e542f3af43a336f66776e42858c381f94b8a1fe92b1ca4b805c5f5 not found: ID does not exist" Oct 30 22:08:56.397717 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.397717 5949 scope.go:122] "RemoveContainer" containerID="056a499f0455153c6c6409a71b6292bf1c957cd5d23143a835b0e63819b58859" Oct 30 22:08:56.397918 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.397900 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"056a499f0455153c6c6409a71b6292bf1c957cd5d23143a835b0e63819b58859"} err="failed to get container status \"056a499f0455153c6c6409a71b6292bf1c957cd5d23143a835b0e63819b58859\": rpc error: code = NotFound desc = could not find container \"056a499f0455153c6c6409a71b6292bf1c957cd5d23143a835b0e63819b58859\": container with ID starting with 056a499f0455153c6c6409a71b6292bf1c957cd5d23143a835b0e63819b58859 not found: ID does not exist" Oct 30 22:08:56.397918 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.397915 5949 scope.go:122] "RemoveContainer" containerID="6df39d2515c9a702cf3a1d64f1cc0a8a9e056bbb0fa98cf0ebaad9dad3096a05" Oct 30 22:08:56.398148 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.398131 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6df39d2515c9a702cf3a1d64f1cc0a8a9e056bbb0fa98cf0ebaad9dad3096a05"} err="failed to get container status \"6df39d2515c9a702cf3a1d64f1cc0a8a9e056bbb0fa98cf0ebaad9dad3096a05\": rpc error: code = NotFound desc = could not find container \"6df39d2515c9a702cf3a1d64f1cc0a8a9e056bbb0fa98cf0ebaad9dad3096a05\": container with ID starting with 6df39d2515c9a702cf3a1d64f1cc0a8a9e056bbb0fa98cf0ebaad9dad3096a05 not found: ID does not exist" Oct 30 22:08:56.398214 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.398149 5949 scope.go:122] "RemoveContainer" containerID="36bd1a5bed3b82469994cb7d98f25114091efa66af5fe091e3037500ac0d995b" Oct 30 22:08:56.398401 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.398383 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36bd1a5bed3b82469994cb7d98f25114091efa66af5fe091e3037500ac0d995b"} err="failed to get container status \"36bd1a5bed3b82469994cb7d98f25114091efa66af5fe091e3037500ac0d995b\": rpc error: code = NotFound desc = could not find container \"36bd1a5bed3b82469994cb7d98f25114091efa66af5fe091e3037500ac0d995b\": container with ID starting with 36bd1a5bed3b82469994cb7d98f25114091efa66af5fe091e3037500ac0d995b not found: ID does not exist" Oct 30 22:08:56.398444 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.398402 5949 scope.go:122] "RemoveContainer" containerID="8b9b1cb2d4b58b1640e7b7ffa96201c957c0f209ed4cbb0101202683c3db5f62" Oct 30 22:08:56.398595 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.398578 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b9b1cb2d4b58b1640e7b7ffa96201c957c0f209ed4cbb0101202683c3db5f62"} err="failed to get container status \"8b9b1cb2d4b58b1640e7b7ffa96201c957c0f209ed4cbb0101202683c3db5f62\": rpc error: code = NotFound desc = could not find container \"8b9b1cb2d4b58b1640e7b7ffa96201c957c0f209ed4cbb0101202683c3db5f62\": container with ID starting with 8b9b1cb2d4b58b1640e7b7ffa96201c957c0f209ed4cbb0101202683c3db5f62 not found: ID does not exist" Oct 30 22:08:56.406192 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.406174 5949 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 30 22:08:56.434431 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.434412 5949 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 30 22:08:56.434755 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.434741 5949 cpu_manager.go:414] "containerMap: removing container" podUID="3913a2c9-e88c-4f49-bc63-02b1fa543c38" containerName="thanos-sidecar" Oct 30 22:08:56.434823 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.434758 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3913a2c9-e88c-4f49-bc63-02b1fa543c38" containerName="thanos-sidecar" Oct 30 22:08:56.434823 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.434783 5949 cpu_manager.go:414] "containerMap: removing container" podUID="3913a2c9-e88c-4f49-bc63-02b1fa543c38" containerName="kube-rbac-proxy" Oct 30 22:08:56.434823 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.434791 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3913a2c9-e88c-4f49-bc63-02b1fa543c38" containerName="kube-rbac-proxy" Oct 30 22:08:56.434823 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.434808 5949 cpu_manager.go:414] "containerMap: removing container" podUID="3913a2c9-e88c-4f49-bc63-02b1fa543c38" containerName="init-config-reloader" Oct 30 22:08:56.434823 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.434815 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3913a2c9-e88c-4f49-bc63-02b1fa543c38" containerName="init-config-reloader" Oct 30 22:08:56.435065 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.434827 5949 cpu_manager.go:414] "containerMap: removing container" podUID="3913a2c9-e88c-4f49-bc63-02b1fa543c38" containerName="kube-rbac-proxy-web" Oct 30 22:08:56.435065 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.434834 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3913a2c9-e88c-4f49-bc63-02b1fa543c38" containerName="kube-rbac-proxy-web" Oct 30 22:08:56.435065 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.434848 5949 cpu_manager.go:414] "containerMap: removing container" podUID="3913a2c9-e88c-4f49-bc63-02b1fa543c38" containerName="prometheus" Oct 30 22:08:56.435065 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.434856 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3913a2c9-e88c-4f49-bc63-02b1fa543c38" containerName="prometheus" Oct 30 22:08:56.435065 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.434876 5949 cpu_manager.go:414] "containerMap: removing container" podUID="3913a2c9-e88c-4f49-bc63-02b1fa543c38" containerName="config-reloader" Oct 30 22:08:56.435065 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.434883 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3913a2c9-e88c-4f49-bc63-02b1fa543c38" containerName="config-reloader" Oct 30 22:08:56.435065 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.434896 5949 cpu_manager.go:414] "containerMap: removing container" podUID="3913a2c9-e88c-4f49-bc63-02b1fa543c38" containerName="kube-rbac-proxy-thanos" Oct 30 22:08:56.435065 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.434903 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3913a2c9-e88c-4f49-bc63-02b1fa543c38" containerName="kube-rbac-proxy-thanos" Oct 30 22:08:56.435065 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.434983 5949 memory_manager.go:365] "RemoveStaleState removing state" podUID="3913a2c9-e88c-4f49-bc63-02b1fa543c38" containerName="config-reloader" Oct 30 22:08:56.435065 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.434994 5949 memory_manager.go:365] "RemoveStaleState removing state" podUID="3913a2c9-e88c-4f49-bc63-02b1fa543c38" containerName="kube-rbac-proxy-thanos" Oct 30 22:08:56.435065 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.435006 5949 memory_manager.go:365] "RemoveStaleState removing state" podUID="3913a2c9-e88c-4f49-bc63-02b1fa543c38" containerName="thanos-sidecar" Oct 30 22:08:56.435065 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.435017 5949 memory_manager.go:365] "RemoveStaleState removing state" podUID="3913a2c9-e88c-4f49-bc63-02b1fa543c38" containerName="kube-rbac-proxy" Oct 30 22:08:56.435065 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.435028 5949 memory_manager.go:365] "RemoveStaleState removing state" podUID="3913a2c9-e88c-4f49-bc63-02b1fa543c38" containerName="init-config-reloader" Oct 30 22:08:56.435065 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.435038 5949 memory_manager.go:365] "RemoveStaleState removing state" podUID="3913a2c9-e88c-4f49-bc63-02b1fa543c38" containerName="kube-rbac-proxy-web" Oct 30 22:08:56.435065 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.435048 5949 memory_manager.go:365] "RemoveStaleState removing state" podUID="3913a2c9-e88c-4f49-bc63-02b1fa543c38" containerName="prometheus" Oct 30 22:08:56.440944 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.440926 5949 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.456263 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.456243 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Oct 30 22:08:56.456464 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.456444 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Oct 30 22:08:56.456561 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.456547 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Oct 30 22:08:56.456598 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.456555 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Oct 30 22:08:56.456753 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.456740 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Oct 30 22:08:56.467589 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.467573 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-p4pd4\"" Oct 30 22:08:56.467651 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.467584 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Oct 30 22:08:56.467749 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.467736 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Oct 30 22:08:56.471720 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.471703 5949 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 30 22:08:56.532199 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.532170 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.532199 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.532202 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.532373 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.532241 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.532373 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.532330 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.532373 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.532357 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.532460 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.532394 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-web-config\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.532498 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.532479 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-config-out\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.532549 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.532537 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.532580 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.532565 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.532612 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.532594 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.532645 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.532629 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.532695 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.532682 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.532748 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.532734 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.532784 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.532756 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.532784 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.532770 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tk7r8\" (UniqueName: \"kubernetes.io/projected/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-kube-api-access-tk7r8\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.532842 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.532818 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.532890 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.532842 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.532890 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.532860 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.532890 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.532879 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-config\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.532973 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.532892 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.556985 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.556961 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Oct 30 22:08:56.556985 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.556981 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Oct 30 22:08:56.557095 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.556991 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Oct 30 22:08:56.557095 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.556961 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Oct 30 22:08:56.557095 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.557024 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Oct 30 22:08:56.557177 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.557139 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-anvv041vkp1fj\"" Oct 30 22:08:56.557233 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.557208 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Oct 30 22:08:56.557947 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.557932 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Oct 30 22:08:56.633605 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.633533 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-config-out\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.633605 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.633588 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.633759 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.633615 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.633759 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.633644 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.633759 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.633667 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.633759 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.633686 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.633759 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.633731 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.633759 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.633756 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.634017 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.633781 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tk7r8\" (UniqueName: \"kubernetes.io/projected/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-kube-api-access-tk7r8\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.634017 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.633811 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.634017 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.633845 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.634017 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.633875 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.634017 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.633907 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-config\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.634017 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.633928 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.634017 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.633985 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.634437 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.634015 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.634437 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.634049 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.634437 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.634127 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.634437 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.634164 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.634437 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.634193 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-web-config\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.634689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.634520 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.634689 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.634667 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.634793 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.634776 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.638084 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.635716 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.638084 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.636659 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.638084 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.636856 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.638084 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.637025 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.638084 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.637107 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-config-out\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.638084 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.637261 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.638084 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.637329 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-web-config\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.638084 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.637477 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.639328 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.638917 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.639328 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.638945 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.639328 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.639075 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.639328 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.639143 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.639328 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.639267 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.639576 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.639540 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-config\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.639667 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.639649 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.640501 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.640478 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.658266 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.658241 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tk7r8\" (UniqueName: \"kubernetes.io/projected/e80ba5e9-a0f2-4303-91b2-41fe096f88fe-kube-api-access-tk7r8\") pod \"prometheus-k8s-0\" (UID: \"e80ba5e9-a0f2-4303-91b2-41fe096f88fe\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:56.766395 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:56.766363 5949 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:08:57.089060 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:57.089024 5949 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 30 22:08:57.092035 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:08:57.092008 5949 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode80ba5e9_a0f2_4303_91b2_41fe096f88fe.slice/crio-9c14e98a2d7df47d48dc8be9872a805620a8bfac2f91e17d14895d98c1895c62 WatchSource:0}: Error finding container 9c14e98a2d7df47d48dc8be9872a805620a8bfac2f91e17d14895d98c1895c62: Status 404 returned error can't find the container with id 9c14e98a2d7df47d48dc8be9872a805620a8bfac2f91e17d14895d98c1895c62 Oct 30 22:08:57.345701 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:57.345607 5949 generic.go:356] "Generic (PLEG): container finished" podID="e80ba5e9-a0f2-4303-91b2-41fe096f88fe" containerID="3fd59ca6103c3e7bd83bc30b99c3a4eefea3faa6fa14f478275e8d02b0bf3796" exitCode=0 Oct 30 22:08:57.345701 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:57.345643 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e80ba5e9-a0f2-4303-91b2-41fe096f88fe","Type":"ContainerDied","Data":"3fd59ca6103c3e7bd83bc30b99c3a4eefea3faa6fa14f478275e8d02b0bf3796"} Oct 30 22:08:57.345701 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:57.345695 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e80ba5e9-a0f2-4303-91b2-41fe096f88fe","Type":"ContainerStarted","Data":"9c14e98a2d7df47d48dc8be9872a805620a8bfac2f91e17d14895d98c1895c62"} Oct 30 22:08:58.064610 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:58.064580 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-zlbzf" Oct 30 22:08:58.160313 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:58.160283 5949 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3913a2c9-e88c-4f49-bc63-02b1fa543c38" path="/var/lib/kubelet/pods/3913a2c9-e88c-4f49-bc63-02b1fa543c38/volumes" Oct 30 22:08:58.188330 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:58.188292 5949 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-6d4659d6dd-rb87s" podUID="94283f0e-a719-4152-aea8-e219f7bb7eef" containerName="console" containerID="cri-o://f2ae40eb21372803272ed17777f5660bced316438dd19549aaa7b0e05d4385dc" gracePeriod=15 Oct 30 22:08:58.351428 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:58.351350 5949 generic.go:356] "Generic (PLEG): container finished" podID="94283f0e-a719-4152-aea8-e219f7bb7eef" containerID="f2ae40eb21372803272ed17777f5660bced316438dd19549aaa7b0e05d4385dc" exitCode=0 Oct 30 22:08:58.351807 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:58.351424 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d4659d6dd-rb87s" event={"ID":"94283f0e-a719-4152-aea8-e219f7bb7eef","Type":"ContainerDied","Data":"f2ae40eb21372803272ed17777f5660bced316438dd19549aaa7b0e05d4385dc"} Oct 30 22:08:58.355930 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:58.355904 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e80ba5e9-a0f2-4303-91b2-41fe096f88fe","Type":"ContainerStarted","Data":"9370bb4e0e51be9d228457ace9f15d81b7fba23d49a2f38204136ef27021fec7"} Oct 30 22:08:58.356035 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:58.355979 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e80ba5e9-a0f2-4303-91b2-41fe096f88fe","Type":"ContainerStarted","Data":"4c311972d043bc7a6be0c38303ef3fe1c0bdef7636c11c3dcf224eb097dee027"} Oct 30 22:08:58.356114 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:58.356054 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e80ba5e9-a0f2-4303-91b2-41fe096f88fe","Type":"ContainerStarted","Data":"32bfd5f8e2fdff5c3716dcda965d4ba97e9cd2d7598c7f2f8532830cb806af3d"} Oct 30 22:08:58.356114 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:58.356097 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e80ba5e9-a0f2-4303-91b2-41fe096f88fe","Type":"ContainerStarted","Data":"aa2e1efacb981b57a89bcc417d92b059b3595658f66352dcdb2488d63c1f0d4c"} Oct 30 22:08:58.356240 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:58.356135 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e80ba5e9-a0f2-4303-91b2-41fe096f88fe","Type":"ContainerStarted","Data":"bf411cb54f8c045b6eee897eb9aa9ef4eef71b19d0cd1ca86a57a49f06d86d98"} Oct 30 22:08:58.356240 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:58.356176 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e80ba5e9-a0f2-4303-91b2-41fe096f88fe","Type":"ContainerStarted","Data":"30d8c535dc5ff74e17c5b4e2f8af0e8f5b1247495b207a42f92908feef6bdcee"} Oct 30 22:08:58.423095 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:58.423045 5949 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.408245683 podStartE2EDuration="2.423032367s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.786684ms" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:58.422617752 +0000 UTC m=+26.386426359" watchObservedRunningTime="2026-10-30 22:08:58.423032367 +0000 UTC m=+26.386840961" Oct 30 22:08:58.428494 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:58.428475 5949 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:58.559142 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:58.559110 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-serving-cert\") pod \"94283f0e-a719-4152-aea8-e219f7bb7eef\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " Oct 30 22:08:58.559308 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:58.559159 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-trusted-ca-bundle\") pod \"94283f0e-a719-4152-aea8-e219f7bb7eef\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " Oct 30 22:08:58.559308 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:58.559198 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-oauth-serving-cert\") pod \"94283f0e-a719-4152-aea8-e219f7bb7eef\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " Oct 30 22:08:58.559411 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:58.559311 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/94283f0e-a719-4152-aea8-e219f7bb7eef-kube-api-access-k2kkt\" (UniqueName: \"kubernetes.io/projected/94283f0e-a719-4152-aea8-e219f7bb7eef-kube-api-access-k2kkt\") pod \"94283f0e-a719-4152-aea8-e219f7bb7eef\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " Oct 30 22:08:58.559411 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:58.559370 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-console-config\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-console-config\") pod \"94283f0e-a719-4152-aea8-e219f7bb7eef\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " Oct 30 22:08:58.559500 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:58.559415 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94283f0e-a719-4152-aea8-e219f7bb7eef-tmp\" (UniqueName: \"kubernetes.io/empty-dir/94283f0e-a719-4152-aea8-e219f7bb7eef-tmp\") pod \"94283f0e-a719-4152-aea8-e219f7bb7eef\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " Oct 30 22:08:58.559500 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:58.559474 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-oauth-config\") pod \"94283f0e-a719-4152-aea8-e219f7bb7eef\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " Oct 30 22:08:58.559600 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:58.559517 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-service-ca\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-service-ca\") pod \"94283f0e-a719-4152-aea8-e219f7bb7eef\" (UID: \"94283f0e-a719-4152-aea8-e219f7bb7eef\") " Oct 30 22:08:58.559600 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:58.559575 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-oauth-serving-cert" pod "94283f0e-a719-4152-aea8-e219f7bb7eef" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:08:58.559733 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:58.559696 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94283f0e-a719-4152-aea8-e219f7bb7eef-tmp" pod "94283f0e-a719-4152-aea8-e219f7bb7eef" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 22:08:58.559808 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:58.559763 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-trusted-ca-bundle" pod "94283f0e-a719-4152-aea8-e219f7bb7eef" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:08:58.559808 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:58.559772 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-console-config" pod "94283f0e-a719-4152-aea8-e219f7bb7eef" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:08:58.559898 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:58.559840 5949 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94283f0e-a719-4152-aea8-e219f7bb7eef-tmp\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:08:58.559898 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:58.559885 5949 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-trusted-ca-bundle\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:08:58.560010 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:58.559900 5949 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-oauth-serving-cert\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:08:58.560010 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:58.559916 5949 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-console-config\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:08:58.560110 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:58.560015 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-service-ca" pod "94283f0e-a719-4152-aea8-e219f7bb7eef" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:08:58.561647 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:58.561620 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-serving-cert" pod "94283f0e-a719-4152-aea8-e219f7bb7eef" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:08:58.561769 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:58.561649 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94283f0e-a719-4152-aea8-e219f7bb7eef-kube-api-access-k2kkt" pod "94283f0e-a719-4152-aea8-e219f7bb7eef" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef"). InnerVolumeSpecName "kube-api-access-k2kkt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 22:08:58.561769 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:58.561747 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-oauth-config" pod "94283f0e-a719-4152-aea8-e219f7bb7eef" (UID: "94283f0e-a719-4152-aea8-e219f7bb7eef"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:08:58.660707 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:58.660686 5949 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-serving-cert\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:08:58.660822 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:58.660709 5949 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2kkt\" (UniqueName: \"kubernetes.io/projected/94283f0e-a719-4152-aea8-e219f7bb7eef-kube-api-access-k2kkt\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:08:58.660822 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:58.660720 5949 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/94283f0e-a719-4152-aea8-e219f7bb7eef-console-oauth-config\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:08:58.660822 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:58.660730 5949 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/94283f0e-a719-4152-aea8-e219f7bb7eef-service-ca\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:08:59.361405 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:59.361361 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d4659d6dd-rb87s" event={"ID":"94283f0e-a719-4152-aea8-e219f7bb7eef","Type":"ContainerDied","Data":"2320e8be4a7af79dbed9ac74a3043708a1694e6a3e6ecd5a21a60e433f61e3b5"} Oct 30 22:08:59.361852 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:59.361434 5949 scope.go:122] "RemoveContainer" containerID="f2ae40eb21372803272ed17777f5660bced316438dd19549aaa7b0e05d4385dc" Oct 30 22:08:59.361852 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:59.361455 5949 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d4659d6dd-rb87s" Oct 30 22:08:59.411599 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:59.411570 5949 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d4659d6dd-rb87s"] Oct 30 22:08:59.423025 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:08:59.422992 5949 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6d4659d6dd-rb87s"] Oct 30 22:09:00.161737 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:09:00.161693 5949 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="94283f0e-a719-4152-aea8-e219f7bb7eef" path="/var/lib/kubelet/pods/94283f0e-a719-4152-aea8-e219f7bb7eef/volumes" Oct 30 22:09:01.104651 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:09:01.104620 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-knzz5" Oct 30 22:09:01.766639 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:09:01.766599 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:09:25.336475 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:09:25.336433 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:09:29.290376 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:09:29.290348 5949 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:09:29.305664 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:09:29.305640 5949 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:09:29.491467 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:09:29.491431 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Oct 30 22:09:55.336076 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:09:55.336037 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:10:25.335514 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:10:25.335474 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:10:55.335786 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:10:55.335747 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:11:25.336050 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:11:25.336016 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:11:55.336204 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:11:55.336121 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:12:25.335407 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:12:25.335369 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:12:55.335547 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:12:55.335511 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:13:25.335123 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:13:25.335082 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:13:32.187747 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:13:32.187717 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ql6lh_074eeae7-932c-4642-8fea-671898135ab1/ovn-acl-logging/0.log" Oct 30 22:13:32.188316 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:13:32.188168 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ql6lh_074eeae7-932c-4642-8fea-671898135ab1/ovn-acl-logging/0.log" Oct 30 22:13:55.335783 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:13:55.335749 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:14:25.334329 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:14:25.334286 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:14:55.335655 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:14:55.335580 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:15:25.334993 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:15:25.334958 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:15:55.335855 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:15:55.335815 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:16:25.335618 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:16:25.335585 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:16:55.335177 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:16:55.335134 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:17:25.335998 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:17:25.335890 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:17:55.335693 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:17:55.335607 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:18:25.335666 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:18:25.335628 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:18:32.213096 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:18:32.213070 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ql6lh_074eeae7-932c-4642-8fea-671898135ab1/ovn-acl-logging/0.log" Oct 30 22:18:32.213676 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:18:32.213208 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ql6lh_074eeae7-932c-4642-8fea-671898135ab1/ovn-acl-logging/0.log" Oct 30 22:18:55.335180 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:18:55.335143 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:19:25.335165 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:19:25.335130 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:19:55.335568 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:19:55.335533 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:19:56.516691 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:19:56.516652 5949 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kserve/s3-init-jtvh9"] Oct 30 22:19:56.517072 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:19:56.516970 5949 cpu_manager.go:414] "containerMap: removing container" podUID="94283f0e-a719-4152-aea8-e219f7bb7eef" containerName="console" Oct 30 22:19:56.517072 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:19:56.516983 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="94283f0e-a719-4152-aea8-e219f7bb7eef" containerName="console" Oct 30 22:19:56.517072 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:19:56.517055 5949 memory_manager.go:365] "RemoveStaleState removing state" podUID="94283f0e-a719-4152-aea8-e219f7bb7eef" containerName="console" Oct 30 22:19:56.520165 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:19:56.520123 5949 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-jtvh9" Oct 30 22:19:56.546117 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:19:56.546070 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"hf-token\"" Oct 30 22:19:56.546295 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:19:56.546274 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\"" Oct 30 22:19:56.546404 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:19:56.546278 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Oct 30 22:19:56.546468 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:19:56.546445 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Oct 30 22:19:56.546752 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:19:56.546729 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-54pjn\"" Oct 30 22:19:56.547426 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:19:56.547402 5949 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-jtvh9"] Oct 30 22:19:56.573419 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:19:56.573383 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c23291be-c2c8-422a-9527-96082ddf12a5-model-cache\") pod \"s3-init-jtvh9\" (UID: \"c23291be-c2c8-422a-9527-96082ddf12a5\") " pod="kserve/s3-init-jtvh9" Oct 30 22:19:56.573563 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:19:56.573447 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbjnf\" (UniqueName: \"kubernetes.io/projected/c23291be-c2c8-422a-9527-96082ddf12a5-kube-api-access-cbjnf\") pod \"s3-init-jtvh9\" (UID: \"c23291be-c2c8-422a-9527-96082ddf12a5\") " pod="kserve/s3-init-jtvh9" Oct 30 22:19:56.674416 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:19:56.674386 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c23291be-c2c8-422a-9527-96082ddf12a5-model-cache\") pod \"s3-init-jtvh9\" (UID: \"c23291be-c2c8-422a-9527-96082ddf12a5\") " pod="kserve/s3-init-jtvh9" Oct 30 22:19:56.674551 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:19:56.674433 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbjnf\" (UniqueName: \"kubernetes.io/projected/c23291be-c2c8-422a-9527-96082ddf12a5-kube-api-access-cbjnf\") pod \"s3-init-jtvh9\" (UID: \"c23291be-c2c8-422a-9527-96082ddf12a5\") " pod="kserve/s3-init-jtvh9" Oct 30 22:19:56.674765 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:19:56.674745 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c23291be-c2c8-422a-9527-96082ddf12a5-model-cache\") pod \"s3-init-jtvh9\" (UID: \"c23291be-c2c8-422a-9527-96082ddf12a5\") " pod="kserve/s3-init-jtvh9" Oct 30 22:19:56.697522 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:19:56.697495 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbjnf\" (UniqueName: \"kubernetes.io/projected/c23291be-c2c8-422a-9527-96082ddf12a5-kube-api-access-cbjnf\") pod \"s3-init-jtvh9\" (UID: \"c23291be-c2c8-422a-9527-96082ddf12a5\") " pod="kserve/s3-init-jtvh9" Oct 30 22:19:56.848192 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:19:56.848112 5949 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-jtvh9" Oct 30 22:19:57.144643 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:19:57.144568 5949 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-jtvh9"] Oct 30 22:19:57.148441 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:19:57.148413 5949 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc23291be_c2c8_422a_9527_96082ddf12a5.slice/crio-579f19c87e6424f06c92ab88335dfc3a694b7563140730dee5187edce64130ed WatchSource:0}: Error finding container 579f19c87e6424f06c92ab88335dfc3a694b7563140730dee5187edce64130ed: Status 404 returned error can't find the container with id 579f19c87e6424f06c92ab88335dfc3a694b7563140730dee5187edce64130ed Oct 30 22:19:57.150321 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:19:57.150303 5949 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 22:19:57.710862 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:19:57.710797 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-jtvh9" event={"ID":"c23291be-c2c8-422a-9527-96082ddf12a5","Type":"ContainerStarted","Data":"579f19c87e6424f06c92ab88335dfc3a694b7563140730dee5187edce64130ed"} Oct 30 22:20:00.724516 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:20:00.724446 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-jtvh9" event={"ID":"c23291be-c2c8-422a-9527-96082ddf12a5","Type":"ContainerStarted","Data":"bb961911c5d5d0b9c1bcbb8885ad3e2f1d742f1de47778bc4d25fea3f9c5cff7"} Oct 30 22:20:02.735838 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:20:02.735800 5949 generic.go:356] "Generic (PLEG): container finished" podID="c23291be-c2c8-422a-9527-96082ddf12a5" containerID="bb961911c5d5d0b9c1bcbb8885ad3e2f1d742f1de47778bc4d25fea3f9c5cff7" exitCode=0 Oct 30 22:20:02.736250 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:20:02.735848 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-jtvh9" event={"ID":"c23291be-c2c8-422a-9527-96082ddf12a5","Type":"ContainerDied","Data":"bb961911c5d5d0b9c1bcbb8885ad3e2f1d742f1de47778bc4d25fea3f9c5cff7"} Oct 30 22:20:07.757846 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:20:07.757810 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-jtvh9" event={"ID":"c23291be-c2c8-422a-9527-96082ddf12a5","Type":"ContainerStarted","Data":"ebf4c28b2abc4d3b43671a59ab2ea17f744d7150ae8e14b46e85363b45bd25d2"} Oct 30 22:20:07.804851 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:20:07.804801 5949 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kserve/s3-init-jtvh9" podStartSLOduration=2.846755929 podStartE2EDuration="11.804786585s" totalImagesPullingTime="7.617496621s" totalInitContainerRuntime="1.340534035s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:19:56 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:20:07.80436086 +0000 UTC m=+695.768169432" watchObservedRunningTime="2026-10-30 22:20:07.804786585 +0000 UTC m=+695.768595152" Oct 30 22:20:25.335488 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:20:25.335448 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:20:55.335616 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:20:55.335536 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:21:11.993794 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:11.993763 5949 generic.go:356] "Generic (PLEG): container finished" podID="c23291be-c2c8-422a-9527-96082ddf12a5" containerID="ebf4c28b2abc4d3b43671a59ab2ea17f744d7150ae8e14b46e85363b45bd25d2" exitCode=0 Oct 30 22:21:11.994156 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:11.993799 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-jtvh9" event={"ID":"c23291be-c2c8-422a-9527-96082ddf12a5","Type":"ContainerDied","Data":"ebf4c28b2abc4d3b43671a59ab2ea17f744d7150ae8e14b46e85363b45bd25d2"} Oct 30 22:21:13.125405 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:13.125380 5949 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-jtvh9" Oct 30 22:21:13.231825 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:13.231788 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c23291be-c2c8-422a-9527-96082ddf12a5-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c23291be-c2c8-422a-9527-96082ddf12a5-model-cache\") pod \"c23291be-c2c8-422a-9527-96082ddf12a5\" (UID: \"c23291be-c2c8-422a-9527-96082ddf12a5\") " Oct 30 22:21:13.231991 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:13.231845 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c23291be-c2c8-422a-9527-96082ddf12a5-kube-api-access-cbjnf\" (UniqueName: \"kubernetes.io/projected/c23291be-c2c8-422a-9527-96082ddf12a5-kube-api-access-cbjnf\") pod \"c23291be-c2c8-422a-9527-96082ddf12a5\" (UID: \"c23291be-c2c8-422a-9527-96082ddf12a5\") " Oct 30 22:21:13.233971 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:13.233946 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c23291be-c2c8-422a-9527-96082ddf12a5-kube-api-access-cbjnf" pod "c23291be-c2c8-422a-9527-96082ddf12a5" (UID: "c23291be-c2c8-422a-9527-96082ddf12a5"). InnerVolumeSpecName "kube-api-access-cbjnf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 22:21:13.252384 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:13.252318 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c23291be-c2c8-422a-9527-96082ddf12a5-model-cache" pod "c23291be-c2c8-422a-9527-96082ddf12a5" (UID: "c23291be-c2c8-422a-9527-96082ddf12a5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 22:21:13.332587 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:13.332558 5949 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c23291be-c2c8-422a-9527-96082ddf12a5-model-cache\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:21:13.332587 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:13.332584 5949 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbjnf\" (UniqueName: \"kubernetes.io/projected/c23291be-c2c8-422a-9527-96082ddf12a5-kube-api-access-cbjnf\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:21:14.002682 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:14.002646 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-jtvh9" event={"ID":"c23291be-c2c8-422a-9527-96082ddf12a5","Type":"ContainerDied","Data":"579f19c87e6424f06c92ab88335dfc3a694b7563140730dee5187edce64130ed"} Oct 30 22:21:14.002682 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:14.002685 5949 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-jtvh9" Oct 30 22:21:14.002882 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:14.002692 5949 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="579f19c87e6424f06c92ab88335dfc3a694b7563140730dee5187edce64130ed" Oct 30 22:21:25.335813 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:21:25.335772 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:21:32.362889 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.362858 5949 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h"] Oct 30 22:21:32.363409 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.363212 5949 cpu_manager.go:414] "containerMap: removing container" podUID="c23291be-c2c8-422a-9527-96082ddf12a5" containerName="s3-init" Oct 30 22:21:32.363409 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.363240 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c23291be-c2c8-422a-9527-96082ddf12a5" containerName="s3-init" Oct 30 22:21:32.363409 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.363296 5949 cpu_manager.go:414] "containerMap: removing container" podUID="c23291be-c2c8-422a-9527-96082ddf12a5" containerName="download-hf-model" Oct 30 22:21:32.363409 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.363302 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c23291be-c2c8-422a-9527-96082ddf12a5" containerName="download-hf-model" Oct 30 22:21:32.363409 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.363366 5949 memory_manager.go:365] "RemoveStaleState removing state" podUID="c23291be-c2c8-422a-9527-96082ddf12a5" containerName="s3-init" Oct 30 22:21:32.366786 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.366769 5949 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" Oct 30 22:21:32.404505 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.404480 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-306b77ff42\"/\"odh-kserve-custom-ca-bundle\"" Oct 30 22:21:32.404677 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.404531 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-306b77ff42\"/\"openshift-service-ca.crt\"" Oct 30 22:21:32.404677 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.404551 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-306b77ff42\"/\"kube-root-ca.crt\"" Oct 30 22:21:32.404832 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.404579 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-306b77ff42\"/\"seaweedfs-s3-creds\"" Oct 30 22:21:32.404832 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.404689 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-306b77ff42\"/\"isvc-canary-rollback-v2-predictor-serving-cert\"" Oct 30 22:21:32.404832 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.404601 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-306b77ff42\"/\"isvc-canary-rollback-kube-rbac-proxy-sar-config\"" Oct 30 22:21:32.404832 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.404756 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-306b77ff42\"/\"default-dockercfg-7rhfl\"" Oct 30 22:21:32.405692 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.405672 5949 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h"] Oct 30 22:21:32.486329 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.486297 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/85870b59-9ca8-47db-ad4e-c57ef471f9ba-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h\" (UID: \"85870b59-9ca8-47db-ad4e-c57ef471f9ba\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" Oct 30 22:21:32.486524 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.486431 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/85870b59-9ca8-47db-ad4e-c57ef471f9ba-proxy-tls\") pod \"isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h\" (UID: \"85870b59-9ca8-47db-ad4e-c57ef471f9ba\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" Oct 30 22:21:32.486524 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.486456 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z876w\" (UniqueName: \"kubernetes.io/projected/85870b59-9ca8-47db-ad4e-c57ef471f9ba-kube-api-access-z876w\") pod \"isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h\" (UID: \"85870b59-9ca8-47db-ad4e-c57ef471f9ba\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" Oct 30 22:21:32.486524 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.486518 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/85870b59-9ca8-47db-ad4e-c57ef471f9ba-kserve-provision-location\") pod \"isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h\" (UID: \"85870b59-9ca8-47db-ad4e-c57ef471f9ba\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" Oct 30 22:21:32.486685 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.486579 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/85870b59-9ca8-47db-ad4e-c57ef471f9ba-cabundle-cert\") pod \"isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h\" (UID: \"85870b59-9ca8-47db-ad4e-c57ef471f9ba\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" Oct 30 22:21:32.504772 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.504742 5949 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk"] Oct 30 22:21:32.508451 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.508430 5949 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" Oct 30 22:21:32.522839 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.522817 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-306b77ff42\"/\"isvc-canary-rollback-predictor-serving-cert\"" Oct 30 22:21:32.531459 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.531434 5949 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk"] Oct 30 22:21:32.587003 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.586971 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/85870b59-9ca8-47db-ad4e-c57ef471f9ba-proxy-tls\") pod \"isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h\" (UID: \"85870b59-9ca8-47db-ad4e-c57ef471f9ba\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" Oct 30 22:21:32.587151 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.587009 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z876w\" (UniqueName: \"kubernetes.io/projected/85870b59-9ca8-47db-ad4e-c57ef471f9ba-kube-api-access-z876w\") pod \"isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h\" (UID: \"85870b59-9ca8-47db-ad4e-c57ef471f9ba\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" Oct 30 22:21:32.587151 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.587027 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/85870b59-9ca8-47db-ad4e-c57ef471f9ba-kserve-provision-location\") pod \"isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h\" (UID: \"85870b59-9ca8-47db-ad4e-c57ef471f9ba\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" Oct 30 22:21:32.587252 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.587150 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pnrl\" (UniqueName: \"kubernetes.io/projected/7e26d533-8597-4a35-91e1-0d5a968fc7c1-kube-api-access-5pnrl\") pod \"isvc-canary-rollback-predictor-786f558f5c-t9jwk\" (UID: \"7e26d533-8597-4a35-91e1-0d5a968fc7c1\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" Oct 30 22:21:32.587252 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.587231 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/85870b59-9ca8-47db-ad4e-c57ef471f9ba-cabundle-cert\") pod \"isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h\" (UID: \"85870b59-9ca8-47db-ad4e-c57ef471f9ba\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" Oct 30 22:21:32.587352 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.587305 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/85870b59-9ca8-47db-ad4e-c57ef471f9ba-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h\" (UID: \"85870b59-9ca8-47db-ad4e-c57ef471f9ba\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" Oct 30 22:21:32.587413 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.587358 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7e26d533-8597-4a35-91e1-0d5a968fc7c1-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-786f558f5c-t9jwk\" (UID: \"7e26d533-8597-4a35-91e1-0d5a968fc7c1\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" Oct 30 22:21:32.587413 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.587400 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7e26d533-8597-4a35-91e1-0d5a968fc7c1-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-786f558f5c-t9jwk\" (UID: \"7e26d533-8597-4a35-91e1-0d5a968fc7c1\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" Oct 30 22:21:32.587508 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.587405 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/85870b59-9ca8-47db-ad4e-c57ef471f9ba-kserve-provision-location\") pod \"isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h\" (UID: \"85870b59-9ca8-47db-ad4e-c57ef471f9ba\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" Oct 30 22:21:32.587508 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.587436 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7e26d533-8597-4a35-91e1-0d5a968fc7c1-proxy-tls\") pod \"isvc-canary-rollback-predictor-786f558f5c-t9jwk\" (UID: \"7e26d533-8597-4a35-91e1-0d5a968fc7c1\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" Oct 30 22:21:32.587621 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.587514 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7e26d533-8597-4a35-91e1-0d5a968fc7c1-cabundle-cert\") pod \"isvc-canary-rollback-predictor-786f558f5c-t9jwk\" (UID: \"7e26d533-8597-4a35-91e1-0d5a968fc7c1\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" Oct 30 22:21:32.587865 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.587845 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/85870b59-9ca8-47db-ad4e-c57ef471f9ba-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h\" (UID: \"85870b59-9ca8-47db-ad4e-c57ef471f9ba\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" Oct 30 22:21:32.587952 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.587927 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/85870b59-9ca8-47db-ad4e-c57ef471f9ba-cabundle-cert\") pod \"isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h\" (UID: \"85870b59-9ca8-47db-ad4e-c57ef471f9ba\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" Oct 30 22:21:32.589453 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.589438 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/85870b59-9ca8-47db-ad4e-c57ef471f9ba-proxy-tls\") pod \"isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h\" (UID: \"85870b59-9ca8-47db-ad4e-c57ef471f9ba\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" Oct 30 22:21:32.609947 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.609923 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z876w\" (UniqueName: \"kubernetes.io/projected/85870b59-9ca8-47db-ad4e-c57ef471f9ba-kube-api-access-z876w\") pod \"isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h\" (UID: \"85870b59-9ca8-47db-ad4e-c57ef471f9ba\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" Oct 30 22:21:32.683637 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.683609 5949 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" Oct 30 22:21:32.688106 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.688088 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7e26d533-8597-4a35-91e1-0d5a968fc7c1-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-786f558f5c-t9jwk\" (UID: \"7e26d533-8597-4a35-91e1-0d5a968fc7c1\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" Oct 30 22:21:32.688180 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.688121 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7e26d533-8597-4a35-91e1-0d5a968fc7c1-proxy-tls\") pod \"isvc-canary-rollback-predictor-786f558f5c-t9jwk\" (UID: \"7e26d533-8597-4a35-91e1-0d5a968fc7c1\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" Oct 30 22:21:32.688180 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.688142 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7e26d533-8597-4a35-91e1-0d5a968fc7c1-cabundle-cert\") pod \"isvc-canary-rollback-predictor-786f558f5c-t9jwk\" (UID: \"7e26d533-8597-4a35-91e1-0d5a968fc7c1\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" Oct 30 22:21:32.688265 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.688196 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pnrl\" (UniqueName: \"kubernetes.io/projected/7e26d533-8597-4a35-91e1-0d5a968fc7c1-kube-api-access-5pnrl\") pod \"isvc-canary-rollback-predictor-786f558f5c-t9jwk\" (UID: \"7e26d533-8597-4a35-91e1-0d5a968fc7c1\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" Oct 30 22:21:32.688312 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.688274 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7e26d533-8597-4a35-91e1-0d5a968fc7c1-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-786f558f5c-t9jwk\" (UID: \"7e26d533-8597-4a35-91e1-0d5a968fc7c1\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" Oct 30 22:21:32.688561 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.688537 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7e26d533-8597-4a35-91e1-0d5a968fc7c1-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-786f558f5c-t9jwk\" (UID: \"7e26d533-8597-4a35-91e1-0d5a968fc7c1\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" Oct 30 22:21:32.688894 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.688875 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7e26d533-8597-4a35-91e1-0d5a968fc7c1-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-786f558f5c-t9jwk\" (UID: \"7e26d533-8597-4a35-91e1-0d5a968fc7c1\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" Oct 30 22:21:32.689192 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.689166 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7e26d533-8597-4a35-91e1-0d5a968fc7c1-cabundle-cert\") pod \"isvc-canary-rollback-predictor-786f558f5c-t9jwk\" (UID: \"7e26d533-8597-4a35-91e1-0d5a968fc7c1\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" Oct 30 22:21:32.690621 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.690595 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7e26d533-8597-4a35-91e1-0d5a968fc7c1-proxy-tls\") pod \"isvc-canary-rollback-predictor-786f558f5c-t9jwk\" (UID: \"7e26d533-8597-4a35-91e1-0d5a968fc7c1\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" Oct 30 22:21:32.717855 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.717821 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pnrl\" (UniqueName: \"kubernetes.io/projected/7e26d533-8597-4a35-91e1-0d5a968fc7c1-kube-api-access-5pnrl\") pod \"isvc-canary-rollback-predictor-786f558f5c-t9jwk\" (UID: \"7e26d533-8597-4a35-91e1-0d5a968fc7c1\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" Oct 30 22:21:32.824963 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:32.824928 5949 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" Oct 30 22:21:33.015177 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:33.015019 5949 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h"] Oct 30 22:21:33.018049 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:21:33.018012 5949 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85870b59_9ca8_47db_ad4e_c57ef471f9ba.slice/crio-ab19426275f8fa1e1c9c576931aa2805fe027608f7a26954a4b0b88eda26d814 WatchSource:0}: Error finding container ab19426275f8fa1e1c9c576931aa2805fe027608f7a26954a4b0b88eda26d814: Status 404 returned error can't find the container with id ab19426275f8fa1e1c9c576931aa2805fe027608f7a26954a4b0b88eda26d814 Oct 30 22:21:33.072400 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:33.072372 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" event={"ID":"85870b59-9ca8-47db-ad4e-c57ef471f9ba","Type":"ContainerStarted","Data":"ab19426275f8fa1e1c9c576931aa2805fe027608f7a26954a4b0b88eda26d814"} Oct 30 22:21:33.118168 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:33.118130 5949 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk"] Oct 30 22:21:33.121589 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:21:33.121563 5949 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e26d533_8597_4a35_91e1_0d5a968fc7c1.slice/crio-361344dbfc4553cb9908b7838d8ba61dbe005ca09e3b4936cde2a1d1da2c3d9f WatchSource:0}: Error finding container 361344dbfc4553cb9908b7838d8ba61dbe005ca09e3b4936cde2a1d1da2c3d9f: Status 404 returned error can't find the container with id 361344dbfc4553cb9908b7838d8ba61dbe005ca09e3b4936cde2a1d1da2c3d9f Oct 30 22:21:34.078507 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:34.078468 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" event={"ID":"7e26d533-8597-4a35-91e1-0d5a968fc7c1","Type":"ContainerStarted","Data":"59dc0faeb80c849794891705ce041578bc8dda1a48d3575e49b85bd51a997ed5"} Oct 30 22:21:34.078938 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:34.078541 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" event={"ID":"7e26d533-8597-4a35-91e1-0d5a968fc7c1","Type":"ContainerStarted","Data":"361344dbfc4553cb9908b7838d8ba61dbe005ca09e3b4936cde2a1d1da2c3d9f"} Oct 30 22:21:34.079802 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:34.079775 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" event={"ID":"85870b59-9ca8-47db-ad4e-c57ef471f9ba","Type":"ContainerStarted","Data":"63e75f61e8a1b020e0fc9d1083c4b6838b494704ce85ff88a1ea3a88c94375ea"} Oct 30 22:21:37.093717 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:37.093628 5949 generic.go:356] "Generic (PLEG): container finished" podID="7e26d533-8597-4a35-91e1-0d5a968fc7c1" containerID="59dc0faeb80c849794891705ce041578bc8dda1a48d3575e49b85bd51a997ed5" exitCode=0 Oct 30 22:21:37.094118 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:37.093715 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" event={"ID":"7e26d533-8597-4a35-91e1-0d5a968fc7c1","Type":"ContainerDied","Data":"59dc0faeb80c849794891705ce041578bc8dda1a48d3575e49b85bd51a997ed5"} Oct 30 22:21:37.095552 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:37.095529 5949 generic.go:356] "Generic (PLEG): container finished" podID="85870b59-9ca8-47db-ad4e-c57ef471f9ba" containerID="63e75f61e8a1b020e0fc9d1083c4b6838b494704ce85ff88a1ea3a88c94375ea" exitCode=0 Oct 30 22:21:37.095657 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:37.095567 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" event={"ID":"85870b59-9ca8-47db-ad4e-c57ef471f9ba","Type":"ContainerDied","Data":"63e75f61e8a1b020e0fc9d1083c4b6838b494704ce85ff88a1ea3a88c94375ea"} Oct 30 22:21:55.342466 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:21:55.342421 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:21:57.220017 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:57.219974 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" event={"ID":"7e26d533-8597-4a35-91e1-0d5a968fc7c1","Type":"ContainerStarted","Data":"49f6c03ac503249a5fd3fb36f13b8b1e3ac4da7e93291145e6c6b490674f2d3f"} Oct 30 22:21:57.222073 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:57.222048 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" event={"ID":"85870b59-9ca8-47db-ad4e-c57ef471f9ba","Type":"ContainerStarted","Data":"62d52db30663091d8685c52d9cc6880fe05886504958e5585c33d4bf52cfdb0d"} Oct 30 22:21:59.232472 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:59.232434 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" event={"ID":"85870b59-9ca8-47db-ad4e-c57ef471f9ba","Type":"ContainerStarted","Data":"5a7c9ac4a98030e1801a521040b487e22c78126f49f661fc5e366e1179d9476b"} Oct 30 22:21:59.232908 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:59.232573 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" Oct 30 22:21:59.237319 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:59.237293 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" event={"ID":"7e26d533-8597-4a35-91e1-0d5a968fc7c1","Type":"ContainerStarted","Data":"cff790484e0f6838b46746901f7d90157a8e9db6c60c165bcbc5082a25ec9eca"} Oct 30 22:21:59.237527 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:59.237504 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" Oct 30 22:21:59.237611 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:59.237553 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" Oct 30 22:21:59.238609 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:59.238581 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" podUID="7e26d533-8597-4a35-91e1-0d5a968fc7c1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused" Oct 30 22:21:59.277016 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:59.276970 5949 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" podStartSLOduration=1.615912538 podStartE2EDuration="27.276957923s" totalImagesPullingTime="21.872666503s" totalInitContainerRuntime="3.788378882s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:21:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:21:59.275792137 +0000 UTC m=+807.239600708" watchObservedRunningTime="2026-10-30 22:21:59.276957923 +0000 UTC m=+807.240766489" Oct 30 22:21:59.325979 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:21:59.325895 5949 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" podStartSLOduration=1.8468990619999999 podStartE2EDuration="27.325882945s" totalImagesPullingTime="21.896993988s" totalInitContainerRuntime="3.581989895s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:21:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:21:59.325843182 +0000 UTC m=+807.289651759" watchObservedRunningTime="2026-10-30 22:21:59.325882945 +0000 UTC m=+807.289691515" Oct 30 22:22:00.241621 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:22:00.241580 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" podUID="7e26d533-8597-4a35-91e1-0d5a968fc7c1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused" Oct 30 22:22:00.242005 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:22:00.241864 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" Oct 30 22:22:00.243075 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:22:00.243050 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" podUID="85870b59-9ca8-47db-ad4e-c57ef471f9ba" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Oct 30 22:22:01.246184 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:22:01.246146 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" podUID="85870b59-9ca8-47db-ad4e-c57ef471f9ba" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Oct 30 22:22:05.246458 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:22:05.246429 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" Oct 30 22:22:05.246933 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:22:05.246905 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" podUID="7e26d533-8597-4a35-91e1-0d5a968fc7c1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused" Oct 30 22:22:06.250798 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:22:06.250764 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" Oct 30 22:22:06.251470 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:22:06.251439 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" podUID="85870b59-9ca8-47db-ad4e-c57ef471f9ba" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Oct 30 22:22:15.247490 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:22:15.247405 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" podUID="7e26d533-8597-4a35-91e1-0d5a968fc7c1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused" Oct 30 22:22:16.252349 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:22:16.252306 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" podUID="85870b59-9ca8-47db-ad4e-c57ef471f9ba" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Oct 30 22:22:25.247583 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:22:25.247536 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" podUID="7e26d533-8597-4a35-91e1-0d5a968fc7c1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused" Oct 30 22:22:25.336262 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:22:25.336150 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:22:26.251927 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:22:26.251894 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" podUID="85870b59-9ca8-47db-ad4e-c57ef471f9ba" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Oct 30 22:22:35.247046 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:22:35.247001 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" podUID="7e26d533-8597-4a35-91e1-0d5a968fc7c1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused" Oct 30 22:22:36.251961 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:22:36.251913 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" podUID="85870b59-9ca8-47db-ad4e-c57ef471f9ba" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Oct 30 22:22:45.246878 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:22:45.246841 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" podUID="7e26d533-8597-4a35-91e1-0d5a968fc7c1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused" Oct 30 22:22:46.251568 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:22:46.251528 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" podUID="85870b59-9ca8-47db-ad4e-c57ef471f9ba" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Oct 30 22:22:55.247938 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:22:55.247873 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" podUID="7e26d533-8597-4a35-91e1-0d5a968fc7c1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused" Oct 30 22:22:55.335325 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:22:55.335291 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:22:56.251638 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:22:56.251598 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" podUID="85870b59-9ca8-47db-ad4e-c57ef471f9ba" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Oct 30 22:23:05.247434 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:05.247396 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" podUID="7e26d533-8597-4a35-91e1-0d5a968fc7c1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused" Oct 30 22:23:06.252358 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:06.252314 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" podUID="85870b59-9ca8-47db-ad4e-c57ef471f9ba" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Oct 30 22:23:08.155903 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:08.155854 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" podUID="85870b59-9ca8-47db-ad4e-c57ef471f9ba" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Oct 30 22:23:15.248375 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:15.248345 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" Oct 30 22:23:18.159422 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:18.159397 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" Oct 30 22:23:18.602805 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:18.602721 5949 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk"] Oct 30 22:23:18.602963 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:18.602937 5949 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" podUID="7e26d533-8597-4a35-91e1-0d5a968fc7c1" containerName="kserve-container" containerID="cri-o://49f6c03ac503249a5fd3fb36f13b8b1e3ac4da7e93291145e6c6b490674f2d3f" gracePeriod=30 Oct 30 22:23:18.603027 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:18.602968 5949 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" podUID="7e26d533-8597-4a35-91e1-0d5a968fc7c1" containerName="kube-rbac-proxy" containerID="cri-o://cff790484e0f6838b46746901f7d90157a8e9db6c60c165bcbc5082a25ec9eca" gracePeriod=30 Oct 30 22:23:19.549400 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:19.549317 5949 generic.go:356] "Generic (PLEG): container finished" podID="7e26d533-8597-4a35-91e1-0d5a968fc7c1" containerID="cff790484e0f6838b46746901f7d90157a8e9db6c60c165bcbc5082a25ec9eca" exitCode=0 Oct 30 22:23:19.549740 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:19.549390 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" event={"ID":"7e26d533-8597-4a35-91e1-0d5a968fc7c1","Type":"ContainerDied","Data":"cff790484e0f6838b46746901f7d90157a8e9db6c60c165bcbc5082a25ec9eca"} Oct 30 22:23:20.242546 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:20.242503 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" podUID="7e26d533-8597-4a35-91e1-0d5a968fc7c1" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.22:8643/healthz\": dial tcp 10.134.0.22:8643: connect: connection refused" Oct 30 22:23:22.213662 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:22.213628 5949 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h"] Oct 30 22:23:22.214126 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:22.213844 5949 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" podUID="85870b59-9ca8-47db-ad4e-c57ef471f9ba" containerName="kserve-container" containerID="cri-o://62d52db30663091d8685c52d9cc6880fe05886504958e5585c33d4bf52cfdb0d" gracePeriod=30 Oct 30 22:23:22.214126 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:22.213880 5949 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" podUID="85870b59-9ca8-47db-ad4e-c57ef471f9ba" containerName="kube-rbac-proxy" containerID="cri-o://5a7c9ac4a98030e1801a521040b487e22c78126f49f661fc5e366e1179d9476b" gracePeriod=30 Oct 30 22:23:22.433608 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:22.433580 5949 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n"] Oct 30 22:23:22.437208 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:22.437189 5949 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" Oct 30 22:23:22.459718 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:22.459694 5949 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n"] Oct 30 22:23:22.531610 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:22.531542 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcv4m\" (UniqueName: \"kubernetes.io/projected/0926c52f-6017-4c2b-a4b0-fd6e26b58a64-kube-api-access-fcv4m\") pod \"isvc-canary-rollback-predictor-786f558f5c-trn4n\" (UID: \"0926c52f-6017-4c2b-a4b0-fd6e26b58a64\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" Oct 30 22:23:22.531610 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:22.531582 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0926c52f-6017-4c2b-a4b0-fd6e26b58a64-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-786f558f5c-trn4n\" (UID: \"0926c52f-6017-4c2b-a4b0-fd6e26b58a64\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" Oct 30 22:23:22.531830 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:22.531635 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/0926c52f-6017-4c2b-a4b0-fd6e26b58a64-cabundle-cert\") pod \"isvc-canary-rollback-predictor-786f558f5c-trn4n\" (UID: \"0926c52f-6017-4c2b-a4b0-fd6e26b58a64\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" Oct 30 22:23:22.531830 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:22.531663 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0926c52f-6017-4c2b-a4b0-fd6e26b58a64-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-786f558f5c-trn4n\" (UID: \"0926c52f-6017-4c2b-a4b0-fd6e26b58a64\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" Oct 30 22:23:22.531830 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:22.531758 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0926c52f-6017-4c2b-a4b0-fd6e26b58a64-proxy-tls\") pod \"isvc-canary-rollback-predictor-786f558f5c-trn4n\" (UID: \"0926c52f-6017-4c2b-a4b0-fd6e26b58a64\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" Oct 30 22:23:22.562929 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:22.562903 5949 generic.go:356] "Generic (PLEG): container finished" podID="85870b59-9ca8-47db-ad4e-c57ef471f9ba" containerID="5a7c9ac4a98030e1801a521040b487e22c78126f49f661fc5e366e1179d9476b" exitCode=0 Oct 30 22:23:22.563051 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:22.562966 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" event={"ID":"85870b59-9ca8-47db-ad4e-c57ef471f9ba","Type":"ContainerDied","Data":"5a7c9ac4a98030e1801a521040b487e22c78126f49f661fc5e366e1179d9476b"} Oct 30 22:23:22.632297 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:22.632268 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0926c52f-6017-4c2b-a4b0-fd6e26b58a64-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-786f558f5c-trn4n\" (UID: \"0926c52f-6017-4c2b-a4b0-fd6e26b58a64\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" Oct 30 22:23:22.632429 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:22.632306 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/0926c52f-6017-4c2b-a4b0-fd6e26b58a64-cabundle-cert\") pod \"isvc-canary-rollback-predictor-786f558f5c-trn4n\" (UID: \"0926c52f-6017-4c2b-a4b0-fd6e26b58a64\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" Oct 30 22:23:22.632429 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:22.632322 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0926c52f-6017-4c2b-a4b0-fd6e26b58a64-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-786f558f5c-trn4n\" (UID: \"0926c52f-6017-4c2b-a4b0-fd6e26b58a64\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" Oct 30 22:23:22.632429 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:22.632363 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0926c52f-6017-4c2b-a4b0-fd6e26b58a64-proxy-tls\") pod \"isvc-canary-rollback-predictor-786f558f5c-trn4n\" (UID: \"0926c52f-6017-4c2b-a4b0-fd6e26b58a64\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" Oct 30 22:23:22.632429 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:22.632393 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcv4m\" (UniqueName: \"kubernetes.io/projected/0926c52f-6017-4c2b-a4b0-fd6e26b58a64-kube-api-access-fcv4m\") pod \"isvc-canary-rollback-predictor-786f558f5c-trn4n\" (UID: \"0926c52f-6017-4c2b-a4b0-fd6e26b58a64\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" Oct 30 22:23:22.632859 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:22.632832 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0926c52f-6017-4c2b-a4b0-fd6e26b58a64-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-786f558f5c-trn4n\" (UID: \"0926c52f-6017-4c2b-a4b0-fd6e26b58a64\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" Oct 30 22:23:22.633022 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:22.633002 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0926c52f-6017-4c2b-a4b0-fd6e26b58a64-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-786f558f5c-trn4n\" (UID: \"0926c52f-6017-4c2b-a4b0-fd6e26b58a64\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" Oct 30 22:23:22.633081 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:22.633067 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/0926c52f-6017-4c2b-a4b0-fd6e26b58a64-cabundle-cert\") pod \"isvc-canary-rollback-predictor-786f558f5c-trn4n\" (UID: \"0926c52f-6017-4c2b-a4b0-fd6e26b58a64\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" Oct 30 22:23:22.634684 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:22.634663 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0926c52f-6017-4c2b-a4b0-fd6e26b58a64-proxy-tls\") pod \"isvc-canary-rollback-predictor-786f558f5c-trn4n\" (UID: \"0926c52f-6017-4c2b-a4b0-fd6e26b58a64\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" Oct 30 22:23:22.655671 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:22.655646 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcv4m\" (UniqueName: \"kubernetes.io/projected/0926c52f-6017-4c2b-a4b0-fd6e26b58a64-kube-api-access-fcv4m\") pod \"isvc-canary-rollback-predictor-786f558f5c-trn4n\" (UID: \"0926c52f-6017-4c2b-a4b0-fd6e26b58a64\") " pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" Oct 30 22:23:22.753835 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:22.753799 5949 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" Oct 30 22:23:23.057074 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:23.057043 5949 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n"] Oct 30 22:23:23.119281 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:23:23.119247 5949 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0926c52f_6017_4c2b_a4b0_fd6e26b58a64.slice/crio-8a7294e320a5c6c19ea8cf8ef48f341f28a3f66c3b717d59e235f25a79701331 WatchSource:0}: Error finding container 8a7294e320a5c6c19ea8cf8ef48f341f28a3f66c3b717d59e235f25a79701331: Status 404 returned error can't find the container with id 8a7294e320a5c6c19ea8cf8ef48f341f28a3f66c3b717d59e235f25a79701331 Oct 30 22:23:23.236741 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:23.236715 5949 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" Oct 30 22:23:23.338931 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:23.338852 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7e26d533-8597-4a35-91e1-0d5a968fc7c1-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7e26d533-8597-4a35-91e1-0d5a968fc7c1-cabundle-cert\") pod \"7e26d533-8597-4a35-91e1-0d5a968fc7c1\" (UID: \"7e26d533-8597-4a35-91e1-0d5a968fc7c1\") " Oct 30 22:23:23.338931 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:23.338897 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e26d533-8597-4a35-91e1-0d5a968fc7c1-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7e26d533-8597-4a35-91e1-0d5a968fc7c1-kserve-provision-location\") pod \"7e26d533-8597-4a35-91e1-0d5a968fc7c1\" (UID: \"7e26d533-8597-4a35-91e1-0d5a968fc7c1\") " Oct 30 22:23:23.338931 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:23.338930 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7e26d533-8597-4a35-91e1-0d5a968fc7c1-proxy-tls\" (UniqueName: \"kubernetes.io/secret/7e26d533-8597-4a35-91e1-0d5a968fc7c1-proxy-tls\") pod \"7e26d533-8597-4a35-91e1-0d5a968fc7c1\" (UID: \"7e26d533-8597-4a35-91e1-0d5a968fc7c1\") " Oct 30 22:23:23.339206 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:23.338961 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7e26d533-8597-4a35-91e1-0d5a968fc7c1-kube-api-access-5pnrl\" (UniqueName: \"kubernetes.io/projected/7e26d533-8597-4a35-91e1-0d5a968fc7c1-kube-api-access-5pnrl\") pod \"7e26d533-8597-4a35-91e1-0d5a968fc7c1\" (UID: \"7e26d533-8597-4a35-91e1-0d5a968fc7c1\") " Oct 30 22:23:23.339206 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:23.338999 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7e26d533-8597-4a35-91e1-0d5a968fc7c1-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7e26d533-8597-4a35-91e1-0d5a968fc7c1-kube-rbac-proxy-sar-config\") pod \"7e26d533-8597-4a35-91e1-0d5a968fc7c1\" (UID: \"7e26d533-8597-4a35-91e1-0d5a968fc7c1\") " Oct 30 22:23:23.339343 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:23.339307 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e26d533-8597-4a35-91e1-0d5a968fc7c1-cabundle-cert" pod "7e26d533-8597-4a35-91e1-0d5a968fc7c1" (UID: "7e26d533-8597-4a35-91e1-0d5a968fc7c1"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:23:23.339387 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:23.339348 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e26d533-8597-4a35-91e1-0d5a968fc7c1-kserve-provision-location" pod "7e26d533-8597-4a35-91e1-0d5a968fc7c1" (UID: "7e26d533-8597-4a35-91e1-0d5a968fc7c1"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 22:23:23.339467 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:23.339447 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e26d533-8597-4a35-91e1-0d5a968fc7c1-kube-rbac-proxy-sar-config" pod "7e26d533-8597-4a35-91e1-0d5a968fc7c1" (UID: "7e26d533-8597-4a35-91e1-0d5a968fc7c1"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:23:23.341252 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:23.341204 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e26d533-8597-4a35-91e1-0d5a968fc7c1-proxy-tls" pod "7e26d533-8597-4a35-91e1-0d5a968fc7c1" (UID: "7e26d533-8597-4a35-91e1-0d5a968fc7c1"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:23:23.341361 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:23.341253 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e26d533-8597-4a35-91e1-0d5a968fc7c1-kube-api-access-5pnrl" pod "7e26d533-8597-4a35-91e1-0d5a968fc7c1" (UID: "7e26d533-8597-4a35-91e1-0d5a968fc7c1"). InnerVolumeSpecName "kube-api-access-5pnrl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 22:23:23.440283 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:23.440254 5949 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7e26d533-8597-4a35-91e1-0d5a968fc7c1-cabundle-cert\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:23:23.440283 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:23.440278 5949 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7e26d533-8597-4a35-91e1-0d5a968fc7c1-kserve-provision-location\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:23:23.440283 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:23.440287 5949 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7e26d533-8597-4a35-91e1-0d5a968fc7c1-proxy-tls\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:23:23.440482 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:23.440297 5949 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pnrl\" (UniqueName: \"kubernetes.io/projected/7e26d533-8597-4a35-91e1-0d5a968fc7c1-kube-api-access-5pnrl\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:23:23.440482 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:23.440306 5949 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7e26d533-8597-4a35-91e1-0d5a968fc7c1-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:23:23.568700 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:23.568659 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" event={"ID":"0926c52f-6017-4c2b-a4b0-fd6e26b58a64","Type":"ContainerStarted","Data":"d1485c23f15de82b9bdd4aac687bf4f81da693d4be0eb112ed80664c927849cf"} Oct 30 22:23:23.568857 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:23.568728 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" event={"ID":"0926c52f-6017-4c2b-a4b0-fd6e26b58a64","Type":"ContainerStarted","Data":"8a7294e320a5c6c19ea8cf8ef48f341f28a3f66c3b717d59e235f25a79701331"} Oct 30 22:23:23.570682 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:23.570657 5949 generic.go:356] "Generic (PLEG): container finished" podID="7e26d533-8597-4a35-91e1-0d5a968fc7c1" containerID="49f6c03ac503249a5fd3fb36f13b8b1e3ac4da7e93291145e6c6b490674f2d3f" exitCode=0 Oct 30 22:23:23.570787 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:23.570688 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" event={"ID":"7e26d533-8597-4a35-91e1-0d5a968fc7c1","Type":"ContainerDied","Data":"49f6c03ac503249a5fd3fb36f13b8b1e3ac4da7e93291145e6c6b490674f2d3f"} Oct 30 22:23:23.570787 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:23.570714 5949 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" Oct 30 22:23:23.570787 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:23.570762 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk" event={"ID":"7e26d533-8597-4a35-91e1-0d5a968fc7c1","Type":"ContainerDied","Data":"361344dbfc4553cb9908b7838d8ba61dbe005ca09e3b4936cde2a1d1da2c3d9f"} Oct 30 22:23:23.570885 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:23.570802 5949 scope.go:122] "RemoveContainer" containerID="cff790484e0f6838b46746901f7d90157a8e9db6c60c165bcbc5082a25ec9eca" Oct 30 22:23:23.579156 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:23.579140 5949 scope.go:122] "RemoveContainer" containerID="49f6c03ac503249a5fd3fb36f13b8b1e3ac4da7e93291145e6c6b490674f2d3f" Oct 30 22:23:23.586943 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:23.586924 5949 scope.go:122] "RemoveContainer" containerID="59dc0faeb80c849794891705ce041578bc8dda1a48d3575e49b85bd51a997ed5" Oct 30 22:23:23.593941 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:23.593923 5949 scope.go:122] "RemoveContainer" containerID="cff790484e0f6838b46746901f7d90157a8e9db6c60c165bcbc5082a25ec9eca" Oct 30 22:23:23.594251 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:23:23.594213 5949 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cff790484e0f6838b46746901f7d90157a8e9db6c60c165bcbc5082a25ec9eca\": container with ID starting with cff790484e0f6838b46746901f7d90157a8e9db6c60c165bcbc5082a25ec9eca not found: ID does not exist" containerID="cff790484e0f6838b46746901f7d90157a8e9db6c60c165bcbc5082a25ec9eca" Oct 30 22:23:23.594336 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:23.594262 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cff790484e0f6838b46746901f7d90157a8e9db6c60c165bcbc5082a25ec9eca"} err="failed to get container status \"cff790484e0f6838b46746901f7d90157a8e9db6c60c165bcbc5082a25ec9eca\": rpc error: code = NotFound desc = could not find container \"cff790484e0f6838b46746901f7d90157a8e9db6c60c165bcbc5082a25ec9eca\": container with ID starting with cff790484e0f6838b46746901f7d90157a8e9db6c60c165bcbc5082a25ec9eca not found: ID does not exist" Oct 30 22:23:23.594336 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:23.594283 5949 scope.go:122] "RemoveContainer" containerID="49f6c03ac503249a5fd3fb36f13b8b1e3ac4da7e93291145e6c6b490674f2d3f" Oct 30 22:23:23.594568 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:23:23.594550 5949 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49f6c03ac503249a5fd3fb36f13b8b1e3ac4da7e93291145e6c6b490674f2d3f\": container with ID starting with 49f6c03ac503249a5fd3fb36f13b8b1e3ac4da7e93291145e6c6b490674f2d3f not found: ID does not exist" containerID="49f6c03ac503249a5fd3fb36f13b8b1e3ac4da7e93291145e6c6b490674f2d3f" Oct 30 22:23:23.594621 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:23.594574 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49f6c03ac503249a5fd3fb36f13b8b1e3ac4da7e93291145e6c6b490674f2d3f"} err="failed to get container status \"49f6c03ac503249a5fd3fb36f13b8b1e3ac4da7e93291145e6c6b490674f2d3f\": rpc error: code = NotFound desc = could not find container \"49f6c03ac503249a5fd3fb36f13b8b1e3ac4da7e93291145e6c6b490674f2d3f\": container with ID starting with 49f6c03ac503249a5fd3fb36f13b8b1e3ac4da7e93291145e6c6b490674f2d3f not found: ID does not exist" Oct 30 22:23:23.594621 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:23.594590 5949 scope.go:122] "RemoveContainer" containerID="59dc0faeb80c849794891705ce041578bc8dda1a48d3575e49b85bd51a997ed5" Oct 30 22:23:23.594828 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:23:23.594813 5949 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59dc0faeb80c849794891705ce041578bc8dda1a48d3575e49b85bd51a997ed5\": container with ID starting with 59dc0faeb80c849794891705ce041578bc8dda1a48d3575e49b85bd51a997ed5 not found: ID does not exist" containerID="59dc0faeb80c849794891705ce041578bc8dda1a48d3575e49b85bd51a997ed5" Oct 30 22:23:23.594874 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:23.594832 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59dc0faeb80c849794891705ce041578bc8dda1a48d3575e49b85bd51a997ed5"} err="failed to get container status \"59dc0faeb80c849794891705ce041578bc8dda1a48d3575e49b85bd51a997ed5\": rpc error: code = NotFound desc = could not find container \"59dc0faeb80c849794891705ce041578bc8dda1a48d3575e49b85bd51a997ed5\": container with ID starting with 59dc0faeb80c849794891705ce041578bc8dda1a48d3575e49b85bd51a997ed5 not found: ID does not exist" Oct 30 22:23:23.646633 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:23.646608 5949 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk"] Oct 30 22:23:23.656239 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:23.652542 5949 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-t9jwk"] Oct 30 22:23:24.159678 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:24.159635 5949 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7e26d533-8597-4a35-91e1-0d5a968fc7c1" path="/var/lib/kubelet/pods/7e26d533-8597-4a35-91e1-0d5a968fc7c1/volumes" Oct 30 22:23:25.335802 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:23:25.335767 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:23:26.247359 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:26.247316 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" podUID="85870b59-9ca8-47db-ad4e-c57ef471f9ba" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.21:8643/healthz\": dial tcp 10.134.0.21:8643: connect: connection refused" Oct 30 22:23:26.855704 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:26.855682 5949 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" Oct 30 22:23:26.967125 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:26.967062 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/85870b59-9ca8-47db-ad4e-c57ef471f9ba-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/85870b59-9ca8-47db-ad4e-c57ef471f9ba-kube-rbac-proxy-sar-config\") pod \"85870b59-9ca8-47db-ad4e-c57ef471f9ba\" (UID: \"85870b59-9ca8-47db-ad4e-c57ef471f9ba\") " Oct 30 22:23:26.967125 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:26.967110 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85870b59-9ca8-47db-ad4e-c57ef471f9ba-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/85870b59-9ca8-47db-ad4e-c57ef471f9ba-kserve-provision-location\") pod \"85870b59-9ca8-47db-ad4e-c57ef471f9ba\" (UID: \"85870b59-9ca8-47db-ad4e-c57ef471f9ba\") " Oct 30 22:23:26.967352 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:26.967253 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/85870b59-9ca8-47db-ad4e-c57ef471f9ba-kube-api-access-z876w\" (UniqueName: \"kubernetes.io/projected/85870b59-9ca8-47db-ad4e-c57ef471f9ba-kube-api-access-z876w\") pod \"85870b59-9ca8-47db-ad4e-c57ef471f9ba\" (UID: \"85870b59-9ca8-47db-ad4e-c57ef471f9ba\") " Oct 30 22:23:26.967352 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:26.967291 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/85870b59-9ca8-47db-ad4e-c57ef471f9ba-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/85870b59-9ca8-47db-ad4e-c57ef471f9ba-cabundle-cert\") pod \"85870b59-9ca8-47db-ad4e-c57ef471f9ba\" (UID: \"85870b59-9ca8-47db-ad4e-c57ef471f9ba\") " Oct 30 22:23:26.967352 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:26.967331 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/85870b59-9ca8-47db-ad4e-c57ef471f9ba-proxy-tls\" (UniqueName: \"kubernetes.io/secret/85870b59-9ca8-47db-ad4e-c57ef471f9ba-proxy-tls\") pod \"85870b59-9ca8-47db-ad4e-c57ef471f9ba\" (UID: \"85870b59-9ca8-47db-ad4e-c57ef471f9ba\") " Oct 30 22:23:26.967510 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:26.967396 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85870b59-9ca8-47db-ad4e-c57ef471f9ba-kserve-provision-location" pod "85870b59-9ca8-47db-ad4e-c57ef471f9ba" (UID: "85870b59-9ca8-47db-ad4e-c57ef471f9ba"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 22:23:26.967510 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:26.967474 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85870b59-9ca8-47db-ad4e-c57ef471f9ba-kube-rbac-proxy-sar-config" pod "85870b59-9ca8-47db-ad4e-c57ef471f9ba" (UID: "85870b59-9ca8-47db-ad4e-c57ef471f9ba"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:23:26.967607 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:26.967589 5949 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/85870b59-9ca8-47db-ad4e-c57ef471f9ba-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:23:26.967656 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:26.967613 5949 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/85870b59-9ca8-47db-ad4e-c57ef471f9ba-kserve-provision-location\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:23:26.967721 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:26.967700 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85870b59-9ca8-47db-ad4e-c57ef471f9ba-cabundle-cert" pod "85870b59-9ca8-47db-ad4e-c57ef471f9ba" (UID: "85870b59-9ca8-47db-ad4e-c57ef471f9ba"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:23:26.969299 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:26.969264 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85870b59-9ca8-47db-ad4e-c57ef471f9ba-kube-api-access-z876w" pod "85870b59-9ca8-47db-ad4e-c57ef471f9ba" (UID: "85870b59-9ca8-47db-ad4e-c57ef471f9ba"). InnerVolumeSpecName "kube-api-access-z876w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 22:23:26.969466 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:26.969447 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85870b59-9ca8-47db-ad4e-c57ef471f9ba-proxy-tls" pod "85870b59-9ca8-47db-ad4e-c57ef471f9ba" (UID: "85870b59-9ca8-47db-ad4e-c57ef471f9ba"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:23:27.068600 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:27.068574 5949 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z876w\" (UniqueName: \"kubernetes.io/projected/85870b59-9ca8-47db-ad4e-c57ef471f9ba-kube-api-access-z876w\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:23:27.068600 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:27.068596 5949 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/85870b59-9ca8-47db-ad4e-c57ef471f9ba-cabundle-cert\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:23:27.068600 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:27.068604 5949 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/85870b59-9ca8-47db-ad4e-c57ef471f9ba-proxy-tls\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:23:27.588447 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:27.588373 5949 generic.go:356] "Generic (PLEG): container finished" podID="0926c52f-6017-4c2b-a4b0-fd6e26b58a64" containerID="d1485c23f15de82b9bdd4aac687bf4f81da693d4be0eb112ed80664c927849cf" exitCode=0 Oct 30 22:23:27.588588 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:27.588446 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" event={"ID":"0926c52f-6017-4c2b-a4b0-fd6e26b58a64","Type":"ContainerDied","Data":"d1485c23f15de82b9bdd4aac687bf4f81da693d4be0eb112ed80664c927849cf"} Oct 30 22:23:27.590453 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:27.590430 5949 generic.go:356] "Generic (PLEG): container finished" podID="85870b59-9ca8-47db-ad4e-c57ef471f9ba" containerID="62d52db30663091d8685c52d9cc6880fe05886504958e5585c33d4bf52cfdb0d" exitCode=0 Oct 30 22:23:27.590549 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:27.590474 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" event={"ID":"85870b59-9ca8-47db-ad4e-c57ef471f9ba","Type":"ContainerDied","Data":"62d52db30663091d8685c52d9cc6880fe05886504958e5585c33d4bf52cfdb0d"} Oct 30 22:23:27.590549 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:27.590484 5949 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" Oct 30 22:23:27.590549 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:27.590517 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h" event={"ID":"85870b59-9ca8-47db-ad4e-c57ef471f9ba","Type":"ContainerDied","Data":"ab19426275f8fa1e1c9c576931aa2805fe027608f7a26954a4b0b88eda26d814"} Oct 30 22:23:27.590549 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:27.590542 5949 scope.go:122] "RemoveContainer" containerID="5a7c9ac4a98030e1801a521040b487e22c78126f49f661fc5e366e1179d9476b" Oct 30 22:23:27.598424 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:27.598406 5949 scope.go:122] "RemoveContainer" containerID="62d52db30663091d8685c52d9cc6880fe05886504958e5585c33d4bf52cfdb0d" Oct 30 22:23:27.605927 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:27.605913 5949 scope.go:122] "RemoveContainer" containerID="63e75f61e8a1b020e0fc9d1083c4b6838b494704ce85ff88a1ea3a88c94375ea" Oct 30 22:23:27.625848 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:27.625830 5949 scope.go:122] "RemoveContainer" containerID="5a7c9ac4a98030e1801a521040b487e22c78126f49f661fc5e366e1179d9476b" Oct 30 22:23:27.626501 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:23:27.626477 5949 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a7c9ac4a98030e1801a521040b487e22c78126f49f661fc5e366e1179d9476b\": container with ID starting with 5a7c9ac4a98030e1801a521040b487e22c78126f49f661fc5e366e1179d9476b not found: ID does not exist" containerID="5a7c9ac4a98030e1801a521040b487e22c78126f49f661fc5e366e1179d9476b" Oct 30 22:23:27.626616 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:27.626511 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a7c9ac4a98030e1801a521040b487e22c78126f49f661fc5e366e1179d9476b"} err="failed to get container status \"5a7c9ac4a98030e1801a521040b487e22c78126f49f661fc5e366e1179d9476b\": rpc error: code = NotFound desc = could not find container \"5a7c9ac4a98030e1801a521040b487e22c78126f49f661fc5e366e1179d9476b\": container with ID starting with 5a7c9ac4a98030e1801a521040b487e22c78126f49f661fc5e366e1179d9476b not found: ID does not exist" Oct 30 22:23:27.626616 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:27.626533 5949 scope.go:122] "RemoveContainer" containerID="62d52db30663091d8685c52d9cc6880fe05886504958e5585c33d4bf52cfdb0d" Oct 30 22:23:27.626840 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:23:27.626824 5949 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62d52db30663091d8685c52d9cc6880fe05886504958e5585c33d4bf52cfdb0d\": container with ID starting with 62d52db30663091d8685c52d9cc6880fe05886504958e5585c33d4bf52cfdb0d not found: ID does not exist" containerID="62d52db30663091d8685c52d9cc6880fe05886504958e5585c33d4bf52cfdb0d" Oct 30 22:23:27.626918 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:27.626856 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62d52db30663091d8685c52d9cc6880fe05886504958e5585c33d4bf52cfdb0d"} err="failed to get container status \"62d52db30663091d8685c52d9cc6880fe05886504958e5585c33d4bf52cfdb0d\": rpc error: code = NotFound desc = could not find container \"62d52db30663091d8685c52d9cc6880fe05886504958e5585c33d4bf52cfdb0d\": container with ID starting with 62d52db30663091d8685c52d9cc6880fe05886504958e5585c33d4bf52cfdb0d not found: ID does not exist" Oct 30 22:23:27.626918 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:27.626876 5949 scope.go:122] "RemoveContainer" containerID="63e75f61e8a1b020e0fc9d1083c4b6838b494704ce85ff88a1ea3a88c94375ea" Oct 30 22:23:27.627181 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:23:27.627162 5949 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63e75f61e8a1b020e0fc9d1083c4b6838b494704ce85ff88a1ea3a88c94375ea\": container with ID starting with 63e75f61e8a1b020e0fc9d1083c4b6838b494704ce85ff88a1ea3a88c94375ea not found: ID does not exist" containerID="63e75f61e8a1b020e0fc9d1083c4b6838b494704ce85ff88a1ea3a88c94375ea" Oct 30 22:23:27.627272 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:27.627188 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63e75f61e8a1b020e0fc9d1083c4b6838b494704ce85ff88a1ea3a88c94375ea"} err="failed to get container status \"63e75f61e8a1b020e0fc9d1083c4b6838b494704ce85ff88a1ea3a88c94375ea\": rpc error: code = NotFound desc = could not find container \"63e75f61e8a1b020e0fc9d1083c4b6838b494704ce85ff88a1ea3a88c94375ea\": container with ID starting with 63e75f61e8a1b020e0fc9d1083c4b6838b494704ce85ff88a1ea3a88c94375ea not found: ID does not exist" Oct 30 22:23:27.679584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:27.679556 5949 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h"] Oct 30 22:23:27.684367 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:27.684345 5949 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-rollback-v2-predictor-57bb7dc8bd-sfb7h"] Oct 30 22:23:28.160213 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:28.160171 5949 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="85870b59-9ca8-47db-ad4e-c57ef471f9ba" path="/var/lib/kubelet/pods/85870b59-9ca8-47db-ad4e-c57ef471f9ba/volumes" Oct 30 22:23:28.596768 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:28.596680 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" event={"ID":"0926c52f-6017-4c2b-a4b0-fd6e26b58a64","Type":"ContainerStarted","Data":"ab071c96c1f5d02082c0aecaf50f7629d1760c97123060e7d28091571548cac5"} Oct 30 22:23:28.596768 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:28.596757 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" event={"ID":"0926c52f-6017-4c2b-a4b0-fd6e26b58a64","Type":"ContainerStarted","Data":"2f164690823b6d610e3f0bad23216a9cbb31ce1dc6ecf0808072821e140d2b57"} Oct 30 22:23:28.596978 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:28.596964 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" Oct 30 22:23:28.652616 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:28.652567 5949 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" podStartSLOduration=2.603144833 podStartE2EDuration="6.652551874s" totalImagesPullingTime="0s" totalInitContainerRuntime="4.049407041s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:23:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:23:28.652090856 +0000 UTC m=+896.615899426" watchObservedRunningTime="2026-10-30 22:23:28.652551874 +0000 UTC m=+896.616360443" Oct 30 22:23:29.602172 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:29.602137 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" Oct 30 22:23:29.603123 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:29.603091 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" podUID="0926c52f-6017-4c2b-a4b0-fd6e26b58a64" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused" Oct 30 22:23:30.605489 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:30.605445 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" podUID="0926c52f-6017-4c2b-a4b0-fd6e26b58a64" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused" Oct 30 22:23:32.242821 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:32.242802 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ql6lh_074eeae7-932c-4642-8fea-671898135ab1/ovn-acl-logging/0.log" Oct 30 22:23:32.244529 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:32.244509 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ql6lh_074eeae7-932c-4642-8fea-671898135ab1/ovn-acl-logging/0.log" Oct 30 22:23:35.609578 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:35.609545 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" Oct 30 22:23:35.610068 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:35.610041 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" podUID="0926c52f-6017-4c2b-a4b0-fd6e26b58a64" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused" Oct 30 22:23:45.610550 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:45.610469 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" podUID="0926c52f-6017-4c2b-a4b0-fd6e26b58a64" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused" Oct 30 22:23:55.336443 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:23:55.336402 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:23:55.610875 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:23:55.610793 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" podUID="0926c52f-6017-4c2b-a4b0-fd6e26b58a64" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused" Oct 30 22:24:05.610539 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:05.610500 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" podUID="0926c52f-6017-4c2b-a4b0-fd6e26b58a64" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused" Oct 30 22:24:15.610617 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:15.610578 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" podUID="0926c52f-6017-4c2b-a4b0-fd6e26b58a64" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused" Oct 30 22:24:25.336287 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:24:25.336252 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:24:25.610093 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:25.610010 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" podUID="0926c52f-6017-4c2b-a4b0-fd6e26b58a64" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused" Oct 30 22:24:35.610340 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:35.610300 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" podUID="0926c52f-6017-4c2b-a4b0-fd6e26b58a64" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused" Oct 30 22:24:41.155006 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:41.154968 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" podUID="0926c52f-6017-4c2b-a4b0-fd6e26b58a64" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused" Oct 30 22:24:51.156170 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:51.156136 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" Oct 30 22:24:52.644740 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:52.644709 5949 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n"] Oct 30 22:24:52.645278 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:52.644963 5949 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" podUID="0926c52f-6017-4c2b-a4b0-fd6e26b58a64" containerName="kserve-container" containerID="cri-o://2f164690823b6d610e3f0bad23216a9cbb31ce1dc6ecf0808072821e140d2b57" gracePeriod=30 Oct 30 22:24:52.645278 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:52.644984 5949 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" podUID="0926c52f-6017-4c2b-a4b0-fd6e26b58a64" containerName="kube-rbac-proxy" containerID="cri-o://ab071c96c1f5d02082c0aecaf50f7629d1760c97123060e7d28091571548cac5" gracePeriod=30 Oct 30 22:24:52.916929 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:52.916841 5949 generic.go:356] "Generic (PLEG): container finished" podID="0926c52f-6017-4c2b-a4b0-fd6e26b58a64" containerID="ab071c96c1f5d02082c0aecaf50f7629d1760c97123060e7d28091571548cac5" exitCode=0 Oct 30 22:24:52.916929 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:52.916906 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" event={"ID":"0926c52f-6017-4c2b-a4b0-fd6e26b58a64","Type":"ContainerDied","Data":"ab071c96c1f5d02082c0aecaf50f7629d1760c97123060e7d28091571548cac5"} Oct 30 22:24:55.335049 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:24:55.335012 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:24:55.605710 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:55.605624 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" podUID="0926c52f-6017-4c2b-a4b0-fd6e26b58a64" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.23:8643/healthz\": dial tcp 10.134.0.23:8643: connect: connection refused" Oct 30 22:24:57.385401 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:57.385376 5949 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" Oct 30 22:24:57.421460 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:57.421393 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0926c52f-6017-4c2b-a4b0-fd6e26b58a64-kube-api-access-fcv4m\" (UniqueName: \"kubernetes.io/projected/0926c52f-6017-4c2b-a4b0-fd6e26b58a64-kube-api-access-fcv4m\") pod \"0926c52f-6017-4c2b-a4b0-fd6e26b58a64\" (UID: \"0926c52f-6017-4c2b-a4b0-fd6e26b58a64\") " Oct 30 22:24:57.421460 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:57.421435 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0926c52f-6017-4c2b-a4b0-fd6e26b58a64-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/0926c52f-6017-4c2b-a4b0-fd6e26b58a64-cabundle-cert\") pod \"0926c52f-6017-4c2b-a4b0-fd6e26b58a64\" (UID: \"0926c52f-6017-4c2b-a4b0-fd6e26b58a64\") " Oct 30 22:24:57.421655 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:57.421472 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0926c52f-6017-4c2b-a4b0-fd6e26b58a64-proxy-tls\" (UniqueName: \"kubernetes.io/secret/0926c52f-6017-4c2b-a4b0-fd6e26b58a64-proxy-tls\") pod \"0926c52f-6017-4c2b-a4b0-fd6e26b58a64\" (UID: \"0926c52f-6017-4c2b-a4b0-fd6e26b58a64\") " Oct 30 22:24:57.421655 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:57.421623 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0926c52f-6017-4c2b-a4b0-fd6e26b58a64-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0926c52f-6017-4c2b-a4b0-fd6e26b58a64-kube-rbac-proxy-sar-config\") pod \"0926c52f-6017-4c2b-a4b0-fd6e26b58a64\" (UID: \"0926c52f-6017-4c2b-a4b0-fd6e26b58a64\") " Oct 30 22:24:57.421772 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:57.421665 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0926c52f-6017-4c2b-a4b0-fd6e26b58a64-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0926c52f-6017-4c2b-a4b0-fd6e26b58a64-kserve-provision-location\") pod \"0926c52f-6017-4c2b-a4b0-fd6e26b58a64\" (UID: \"0926c52f-6017-4c2b-a4b0-fd6e26b58a64\") " Oct 30 22:24:57.421901 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:57.421795 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0926c52f-6017-4c2b-a4b0-fd6e26b58a64-cabundle-cert" pod "0926c52f-6017-4c2b-a4b0-fd6e26b58a64" (UID: "0926c52f-6017-4c2b-a4b0-fd6e26b58a64"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:24:57.421982 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:57.421966 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0926c52f-6017-4c2b-a4b0-fd6e26b58a64-kube-rbac-proxy-sar-config" pod "0926c52f-6017-4c2b-a4b0-fd6e26b58a64" (UID: "0926c52f-6017-4c2b-a4b0-fd6e26b58a64"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:24:57.422039 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:57.421996 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0926c52f-6017-4c2b-a4b0-fd6e26b58a64-kserve-provision-location" pod "0926c52f-6017-4c2b-a4b0-fd6e26b58a64" (UID: "0926c52f-6017-4c2b-a4b0-fd6e26b58a64"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 22:24:57.423644 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:57.423622 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0926c52f-6017-4c2b-a4b0-fd6e26b58a64-proxy-tls" pod "0926c52f-6017-4c2b-a4b0-fd6e26b58a64" (UID: "0926c52f-6017-4c2b-a4b0-fd6e26b58a64"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:24:57.423754 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:57.423676 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0926c52f-6017-4c2b-a4b0-fd6e26b58a64-kube-api-access-fcv4m" pod "0926c52f-6017-4c2b-a4b0-fd6e26b58a64" (UID: "0926c52f-6017-4c2b-a4b0-fd6e26b58a64"). InnerVolumeSpecName "kube-api-access-fcv4m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 22:24:57.523186 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:57.523154 5949 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fcv4m\" (UniqueName: \"kubernetes.io/projected/0926c52f-6017-4c2b-a4b0-fd6e26b58a64-kube-api-access-fcv4m\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:24:57.523186 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:57.523181 5949 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/0926c52f-6017-4c2b-a4b0-fd6e26b58a64-cabundle-cert\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:24:57.523186 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:57.523194 5949 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0926c52f-6017-4c2b-a4b0-fd6e26b58a64-proxy-tls\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:24:57.523430 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:57.523205 5949 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0926c52f-6017-4c2b-a4b0-fd6e26b58a64-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:24:57.523430 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:57.523239 5949 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0926c52f-6017-4c2b-a4b0-fd6e26b58a64-kserve-provision-location\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:24:57.938830 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:57.938795 5949 generic.go:356] "Generic (PLEG): container finished" podID="0926c52f-6017-4c2b-a4b0-fd6e26b58a64" containerID="2f164690823b6d610e3f0bad23216a9cbb31ce1dc6ecf0808072821e140d2b57" exitCode=0 Oct 30 22:24:57.938996 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:57.938852 5949 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" Oct 30 22:24:57.938996 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:57.938879 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" event={"ID":"0926c52f-6017-4c2b-a4b0-fd6e26b58a64","Type":"ContainerDied","Data":"2f164690823b6d610e3f0bad23216a9cbb31ce1dc6ecf0808072821e140d2b57"} Oct 30 22:24:57.938996 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:57.938948 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n" event={"ID":"0926c52f-6017-4c2b-a4b0-fd6e26b58a64","Type":"ContainerDied","Data":"8a7294e320a5c6c19ea8cf8ef48f341f28a3f66c3b717d59e235f25a79701331"} Oct 30 22:24:57.938996 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:57.938986 5949 scope.go:122] "RemoveContainer" containerID="ab071c96c1f5d02082c0aecaf50f7629d1760c97123060e7d28091571548cac5" Oct 30 22:24:57.948396 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:57.948371 5949 scope.go:122] "RemoveContainer" containerID="2f164690823b6d610e3f0bad23216a9cbb31ce1dc6ecf0808072821e140d2b57" Oct 30 22:24:57.957755 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:57.957736 5949 scope.go:122] "RemoveContainer" containerID="d1485c23f15de82b9bdd4aac687bf4f81da693d4be0eb112ed80664c927849cf" Oct 30 22:24:57.964855 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:57.964838 5949 scope.go:122] "RemoveContainer" containerID="ab071c96c1f5d02082c0aecaf50f7629d1760c97123060e7d28091571548cac5" Oct 30 22:24:57.965088 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:24:57.965071 5949 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab071c96c1f5d02082c0aecaf50f7629d1760c97123060e7d28091571548cac5\": container with ID starting with ab071c96c1f5d02082c0aecaf50f7629d1760c97123060e7d28091571548cac5 not found: ID does not exist" containerID="ab071c96c1f5d02082c0aecaf50f7629d1760c97123060e7d28091571548cac5" Oct 30 22:24:57.965151 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:57.965097 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab071c96c1f5d02082c0aecaf50f7629d1760c97123060e7d28091571548cac5"} err="failed to get container status \"ab071c96c1f5d02082c0aecaf50f7629d1760c97123060e7d28091571548cac5\": rpc error: code = NotFound desc = could not find container \"ab071c96c1f5d02082c0aecaf50f7629d1760c97123060e7d28091571548cac5\": container with ID starting with ab071c96c1f5d02082c0aecaf50f7629d1760c97123060e7d28091571548cac5 not found: ID does not exist" Oct 30 22:24:57.965151 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:57.965116 5949 scope.go:122] "RemoveContainer" containerID="2f164690823b6d610e3f0bad23216a9cbb31ce1dc6ecf0808072821e140d2b57" Oct 30 22:24:57.965402 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:24:57.965378 5949 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f164690823b6d610e3f0bad23216a9cbb31ce1dc6ecf0808072821e140d2b57\": container with ID starting with 2f164690823b6d610e3f0bad23216a9cbb31ce1dc6ecf0808072821e140d2b57 not found: ID does not exist" containerID="2f164690823b6d610e3f0bad23216a9cbb31ce1dc6ecf0808072821e140d2b57" Oct 30 22:24:57.965454 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:57.965418 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f164690823b6d610e3f0bad23216a9cbb31ce1dc6ecf0808072821e140d2b57"} err="failed to get container status \"2f164690823b6d610e3f0bad23216a9cbb31ce1dc6ecf0808072821e140d2b57\": rpc error: code = NotFound desc = could not find container \"2f164690823b6d610e3f0bad23216a9cbb31ce1dc6ecf0808072821e140d2b57\": container with ID starting with 2f164690823b6d610e3f0bad23216a9cbb31ce1dc6ecf0808072821e140d2b57 not found: ID does not exist" Oct 30 22:24:57.965454 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:57.965444 5949 scope.go:122] "RemoveContainer" containerID="d1485c23f15de82b9bdd4aac687bf4f81da693d4be0eb112ed80664c927849cf" Oct 30 22:24:57.965739 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:24:57.965719 5949 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1485c23f15de82b9bdd4aac687bf4f81da693d4be0eb112ed80664c927849cf\": container with ID starting with d1485c23f15de82b9bdd4aac687bf4f81da693d4be0eb112ed80664c927849cf not found: ID does not exist" containerID="d1485c23f15de82b9bdd4aac687bf4f81da693d4be0eb112ed80664c927849cf" Oct 30 22:24:57.965796 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:57.965745 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1485c23f15de82b9bdd4aac687bf4f81da693d4be0eb112ed80664c927849cf"} err="failed to get container status \"d1485c23f15de82b9bdd4aac687bf4f81da693d4be0eb112ed80664c927849cf\": rpc error: code = NotFound desc = could not find container \"d1485c23f15de82b9bdd4aac687bf4f81da693d4be0eb112ed80664c927849cf\": container with ID starting with d1485c23f15de82b9bdd4aac687bf4f81da693d4be0eb112ed80664c927849cf not found: ID does not exist" Oct 30 22:24:58.000711 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:58.000679 5949 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n"] Oct 30 22:24:58.012161 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:58.012138 5949 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-rollback-predictor-786f558f5c-trn4n"] Oct 30 22:24:58.159369 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:24:58.159342 5949 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0926c52f-6017-4c2b-a4b0-fd6e26b58a64" path="/var/lib/kubelet/pods/0926c52f-6017-4c2b-a4b0-fd6e26b58a64/volumes" Oct 30 22:25:00.387843 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.387809 5949 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5"] Oct 30 22:25:00.388243 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.388133 5949 cpu_manager.go:414] "containerMap: removing container" podUID="0926c52f-6017-4c2b-a4b0-fd6e26b58a64" containerName="storage-initializer" Oct 30 22:25:00.388243 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.388144 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0926c52f-6017-4c2b-a4b0-fd6e26b58a64" containerName="storage-initializer" Oct 30 22:25:00.388243 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.388155 5949 cpu_manager.go:414] "containerMap: removing container" podUID="0926c52f-6017-4c2b-a4b0-fd6e26b58a64" containerName="kserve-container" Oct 30 22:25:00.388243 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.388160 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0926c52f-6017-4c2b-a4b0-fd6e26b58a64" containerName="kserve-container" Oct 30 22:25:00.388243 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.388168 5949 cpu_manager.go:414] "containerMap: removing container" podUID="85870b59-9ca8-47db-ad4e-c57ef471f9ba" containerName="kube-rbac-proxy" Oct 30 22:25:00.388243 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.388173 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="85870b59-9ca8-47db-ad4e-c57ef471f9ba" containerName="kube-rbac-proxy" Oct 30 22:25:00.388243 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.388187 5949 cpu_manager.go:414] "containerMap: removing container" podUID="7e26d533-8597-4a35-91e1-0d5a968fc7c1" containerName="storage-initializer" Oct 30 22:25:00.388243 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.388192 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7e26d533-8597-4a35-91e1-0d5a968fc7c1" containerName="storage-initializer" Oct 30 22:25:00.388243 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.388200 5949 cpu_manager.go:414] "containerMap: removing container" podUID="85870b59-9ca8-47db-ad4e-c57ef471f9ba" containerName="kserve-container" Oct 30 22:25:00.388243 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.388204 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="85870b59-9ca8-47db-ad4e-c57ef471f9ba" containerName="kserve-container" Oct 30 22:25:00.388243 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.388211 5949 cpu_manager.go:414] "containerMap: removing container" podUID="0926c52f-6017-4c2b-a4b0-fd6e26b58a64" containerName="kube-rbac-proxy" Oct 30 22:25:00.388243 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.388236 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0926c52f-6017-4c2b-a4b0-fd6e26b58a64" containerName="kube-rbac-proxy" Oct 30 22:25:00.388243 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.388256 5949 cpu_manager.go:414] "containerMap: removing container" podUID="7e26d533-8597-4a35-91e1-0d5a968fc7c1" containerName="kserve-container" Oct 30 22:25:00.388638 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.388261 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7e26d533-8597-4a35-91e1-0d5a968fc7c1" containerName="kserve-container" Oct 30 22:25:00.388638 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.388271 5949 cpu_manager.go:414] "containerMap: removing container" podUID="85870b59-9ca8-47db-ad4e-c57ef471f9ba" containerName="storage-initializer" Oct 30 22:25:00.388638 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.388276 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="85870b59-9ca8-47db-ad4e-c57ef471f9ba" containerName="storage-initializer" Oct 30 22:25:00.388638 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.388282 5949 cpu_manager.go:414] "containerMap: removing container" podUID="7e26d533-8597-4a35-91e1-0d5a968fc7c1" containerName="kube-rbac-proxy" Oct 30 22:25:00.388638 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.388286 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7e26d533-8597-4a35-91e1-0d5a968fc7c1" containerName="kube-rbac-proxy" Oct 30 22:25:00.388638 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.388328 5949 memory_manager.go:365] "RemoveStaleState removing state" podUID="85870b59-9ca8-47db-ad4e-c57ef471f9ba" containerName="kserve-container" Oct 30 22:25:00.388638 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.388336 5949 memory_manager.go:365] "RemoveStaleState removing state" podUID="7e26d533-8597-4a35-91e1-0d5a968fc7c1" containerName="kserve-container" Oct 30 22:25:00.388638 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.388343 5949 memory_manager.go:365] "RemoveStaleState removing state" podUID="7e26d533-8597-4a35-91e1-0d5a968fc7c1" containerName="kube-rbac-proxy" Oct 30 22:25:00.388638 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.388352 5949 memory_manager.go:365] "RemoveStaleState removing state" podUID="0926c52f-6017-4c2b-a4b0-fd6e26b58a64" containerName="kserve-container" Oct 30 22:25:00.388638 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.388357 5949 memory_manager.go:365] "RemoveStaleState removing state" podUID="85870b59-9ca8-47db-ad4e-c57ef471f9ba" containerName="kube-rbac-proxy" Oct 30 22:25:00.388638 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.388363 5949 memory_manager.go:365] "RemoveStaleState removing state" podUID="0926c52f-6017-4c2b-a4b0-fd6e26b58a64" containerName="kube-rbac-proxy" Oct 30 22:25:00.392899 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.392877 5949 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" Oct 30 22:25:00.408628 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.408598 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-306b77ff42\"/\"odh-kserve-custom-ca-bundle\"" Oct 30 22:25:00.408731 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.408633 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-306b77ff42\"/\"openshift-service-ca.crt\"" Oct 30 22:25:00.408731 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.408635 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-306b77ff42\"/\"kube-root-ca.crt\"" Oct 30 22:25:00.408731 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.408652 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-306b77ff42\"/\"isvc-canary-stop-kube-rbac-proxy-sar-config\"" Oct 30 22:25:00.419651 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.419631 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-306b77ff42\"/\"seaweedfs-s3-creds\"" Oct 30 22:25:00.419717 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.419696 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-306b77ff42\"/\"default-dockercfg-7rhfl\"" Oct 30 22:25:00.419792 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.419778 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-306b77ff42\"/\"isvc-canary-stop-v2-predictor-serving-cert\"" Oct 30 22:25:00.421570 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.421549 5949 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5"] Oct 30 22:25:00.448735 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.448715 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7834721e-c7e2-4c8e-9d6b-96ab1ce496e5-cabundle-cert\") pod \"isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5\" (UID: \"7834721e-c7e2-4c8e-9d6b-96ab1ce496e5\") " pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" Oct 30 22:25:00.448851 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.448744 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7834721e-c7e2-4c8e-9d6b-96ab1ce496e5-proxy-tls\") pod \"isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5\" (UID: \"7834721e-c7e2-4c8e-9d6b-96ab1ce496e5\") " pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" Oct 30 22:25:00.448851 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.448774 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rrfm\" (UniqueName: \"kubernetes.io/projected/7834721e-c7e2-4c8e-9d6b-96ab1ce496e5-kube-api-access-2rrfm\") pod \"isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5\" (UID: \"7834721e-c7e2-4c8e-9d6b-96ab1ce496e5\") " pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" Oct 30 22:25:00.448851 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.448793 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7834721e-c7e2-4c8e-9d6b-96ab1ce496e5-kserve-provision-location\") pod \"isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5\" (UID: \"7834721e-c7e2-4c8e-9d6b-96ab1ce496e5\") " pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" Oct 30 22:25:00.449020 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.448944 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7834721e-c7e2-4c8e-9d6b-96ab1ce496e5-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5\" (UID: \"7834721e-c7e2-4c8e-9d6b-96ab1ce496e5\") " pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" Oct 30 22:25:00.549559 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.549536 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7834721e-c7e2-4c8e-9d6b-96ab1ce496e5-cabundle-cert\") pod \"isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5\" (UID: \"7834721e-c7e2-4c8e-9d6b-96ab1ce496e5\") " pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" Oct 30 22:25:00.549685 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.549569 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7834721e-c7e2-4c8e-9d6b-96ab1ce496e5-proxy-tls\") pod \"isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5\" (UID: \"7834721e-c7e2-4c8e-9d6b-96ab1ce496e5\") " pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" Oct 30 22:25:00.549685 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.549594 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rrfm\" (UniqueName: \"kubernetes.io/projected/7834721e-c7e2-4c8e-9d6b-96ab1ce496e5-kube-api-access-2rrfm\") pod \"isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5\" (UID: \"7834721e-c7e2-4c8e-9d6b-96ab1ce496e5\") " pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" Oct 30 22:25:00.549791 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.549700 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7834721e-c7e2-4c8e-9d6b-96ab1ce496e5-kserve-provision-location\") pod \"isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5\" (UID: \"7834721e-c7e2-4c8e-9d6b-96ab1ce496e5\") " pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" Oct 30 22:25:00.549791 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.549737 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7834721e-c7e2-4c8e-9d6b-96ab1ce496e5-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5\" (UID: \"7834721e-c7e2-4c8e-9d6b-96ab1ce496e5\") " pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" Oct 30 22:25:00.550094 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.550073 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7834721e-c7e2-4c8e-9d6b-96ab1ce496e5-kserve-provision-location\") pod \"isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5\" (UID: \"7834721e-c7e2-4c8e-9d6b-96ab1ce496e5\") " pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" Oct 30 22:25:00.550313 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.550293 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7834721e-c7e2-4c8e-9d6b-96ab1ce496e5-cabundle-cert\") pod \"isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5\" (UID: \"7834721e-c7e2-4c8e-9d6b-96ab1ce496e5\") " pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" Oct 30 22:25:00.550384 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.550368 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7834721e-c7e2-4c8e-9d6b-96ab1ce496e5-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5\" (UID: \"7834721e-c7e2-4c8e-9d6b-96ab1ce496e5\") " pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" Oct 30 22:25:00.551941 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.551912 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7834721e-c7e2-4c8e-9d6b-96ab1ce496e5-proxy-tls\") pod \"isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5\" (UID: \"7834721e-c7e2-4c8e-9d6b-96ab1ce496e5\") " pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" Oct 30 22:25:00.572567 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.572544 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rrfm\" (UniqueName: \"kubernetes.io/projected/7834721e-c7e2-4c8e-9d6b-96ab1ce496e5-kube-api-access-2rrfm\") pod \"isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5\" (UID: \"7834721e-c7e2-4c8e-9d6b-96ab1ce496e5\") " pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" Oct 30 22:25:00.706066 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.706040 5949 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6"] Oct 30 22:25:00.709477 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.709456 5949 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" Oct 30 22:25:00.709857 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.709842 5949 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" Oct 30 22:25:00.724561 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.724536 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-306b77ff42\"/\"isvc-canary-stop-predictor-serving-cert\"" Oct 30 22:25:00.735285 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.735264 5949 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6"] Oct 30 22:25:00.750720 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.750692 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a939103f-a5b6-455e-a716-eaab00eee319-proxy-tls\") pod \"isvc-canary-stop-predictor-6bd757b497-478x6\" (UID: \"a939103f-a5b6-455e-a716-eaab00eee319\") " pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" Oct 30 22:25:00.750854 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.750722 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55z9j\" (UniqueName: \"kubernetes.io/projected/a939103f-a5b6-455e-a716-eaab00eee319-kube-api-access-55z9j\") pod \"isvc-canary-stop-predictor-6bd757b497-478x6\" (UID: \"a939103f-a5b6-455e-a716-eaab00eee319\") " pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" Oct 30 22:25:00.750854 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.750741 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a939103f-a5b6-455e-a716-eaab00eee319-cabundle-cert\") pod \"isvc-canary-stop-predictor-6bd757b497-478x6\" (UID: \"a939103f-a5b6-455e-a716-eaab00eee319\") " pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" Oct 30 22:25:00.750986 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.750931 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a939103f-a5b6-455e-a716-eaab00eee319-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-predictor-6bd757b497-478x6\" (UID: \"a939103f-a5b6-455e-a716-eaab00eee319\") " pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" Oct 30 22:25:00.751052 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.751025 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a939103f-a5b6-455e-a716-eaab00eee319-kserve-provision-location\") pod \"isvc-canary-stop-predictor-6bd757b497-478x6\" (UID: \"a939103f-a5b6-455e-a716-eaab00eee319\") " pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" Oct 30 22:25:00.851602 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.851564 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a939103f-a5b6-455e-a716-eaab00eee319-proxy-tls\") pod \"isvc-canary-stop-predictor-6bd757b497-478x6\" (UID: \"a939103f-a5b6-455e-a716-eaab00eee319\") " pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" Oct 30 22:25:00.851602 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.851604 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55z9j\" (UniqueName: \"kubernetes.io/projected/a939103f-a5b6-455e-a716-eaab00eee319-kube-api-access-55z9j\") pod \"isvc-canary-stop-predictor-6bd757b497-478x6\" (UID: \"a939103f-a5b6-455e-a716-eaab00eee319\") " pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" Oct 30 22:25:00.851841 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.851623 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a939103f-a5b6-455e-a716-eaab00eee319-cabundle-cert\") pod \"isvc-canary-stop-predictor-6bd757b497-478x6\" (UID: \"a939103f-a5b6-455e-a716-eaab00eee319\") " pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" Oct 30 22:25:00.851841 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.851663 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a939103f-a5b6-455e-a716-eaab00eee319-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-predictor-6bd757b497-478x6\" (UID: \"a939103f-a5b6-455e-a716-eaab00eee319\") " pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" Oct 30 22:25:00.851841 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.851688 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a939103f-a5b6-455e-a716-eaab00eee319-kserve-provision-location\") pod \"isvc-canary-stop-predictor-6bd757b497-478x6\" (UID: \"a939103f-a5b6-455e-a716-eaab00eee319\") " pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" Oct 30 22:25:00.852106 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.852087 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a939103f-a5b6-455e-a716-eaab00eee319-kserve-provision-location\") pod \"isvc-canary-stop-predictor-6bd757b497-478x6\" (UID: \"a939103f-a5b6-455e-a716-eaab00eee319\") " pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" Oct 30 22:25:00.852443 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.852420 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a939103f-a5b6-455e-a716-eaab00eee319-cabundle-cert\") pod \"isvc-canary-stop-predictor-6bd757b497-478x6\" (UID: \"a939103f-a5b6-455e-a716-eaab00eee319\") " pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" Oct 30 22:25:00.852519 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.852444 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a939103f-a5b6-455e-a716-eaab00eee319-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-predictor-6bd757b497-478x6\" (UID: \"a939103f-a5b6-455e-a716-eaab00eee319\") " pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" Oct 30 22:25:00.854053 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.854035 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a939103f-a5b6-455e-a716-eaab00eee319-proxy-tls\") pod \"isvc-canary-stop-predictor-6bd757b497-478x6\" (UID: \"a939103f-a5b6-455e-a716-eaab00eee319\") " pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" Oct 30 22:25:00.874741 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:00.874712 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55z9j\" (UniqueName: \"kubernetes.io/projected/a939103f-a5b6-455e-a716-eaab00eee319-kube-api-access-55z9j\") pod \"isvc-canary-stop-predictor-6bd757b497-478x6\" (UID: \"a939103f-a5b6-455e-a716-eaab00eee319\") " pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" Oct 30 22:25:01.018692 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:01.018614 5949 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5"] Oct 30 22:25:01.022176 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:25:01.022150 5949 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7834721e_c7e2_4c8e_9d6b_96ab1ce496e5.slice/crio-0d91750d050f471be4880c24d56c22443500e451b0cd5b8c6d4bf1912fa86995 WatchSource:0}: Error finding container 0d91750d050f471be4880c24d56c22443500e451b0cd5b8c6d4bf1912fa86995: Status 404 returned error can't find the container with id 0d91750d050f471be4880c24d56c22443500e451b0cd5b8c6d4bf1912fa86995 Oct 30 22:25:01.039998 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:01.039976 5949 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" Oct 30 22:25:01.345592 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:01.345517 5949 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6"] Oct 30 22:25:01.349076 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:25:01.349043 5949 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda939103f_a5b6_455e_a716_eaab00eee319.slice/crio-260afd5ccabba36dcced75f5fb4f04b2c97ec721962334193316eb309c2611b2 WatchSource:0}: Error finding container 260afd5ccabba36dcced75f5fb4f04b2c97ec721962334193316eb309c2611b2: Status 404 returned error can't find the container with id 260afd5ccabba36dcced75f5fb4f04b2c97ec721962334193316eb309c2611b2 Oct 30 22:25:01.957168 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:01.957127 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" event={"ID":"7834721e-c7e2-4c8e-9d6b-96ab1ce496e5","Type":"ContainerStarted","Data":"02065ee61f4d5ab13d73167ee19153b1554f92da0e765c28cc7b72c8c6cccf16"} Oct 30 22:25:01.957637 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:01.957244 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" event={"ID":"7834721e-c7e2-4c8e-9d6b-96ab1ce496e5","Type":"ContainerStarted","Data":"0d91750d050f471be4880c24d56c22443500e451b0cd5b8c6d4bf1912fa86995"} Oct 30 22:25:01.958864 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:01.958839 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" event={"ID":"a939103f-a5b6-455e-a716-eaab00eee319","Type":"ContainerStarted","Data":"73d555f6a71d8fbb3068a9f12c16f47df3aeaa8883a21097130603346a929903"} Oct 30 22:25:01.958959 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:01.958886 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" event={"ID":"a939103f-a5b6-455e-a716-eaab00eee319","Type":"ContainerStarted","Data":"260afd5ccabba36dcced75f5fb4f04b2c97ec721962334193316eb309c2611b2"} Oct 30 22:25:05.976550 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:05.976516 5949 generic.go:356] "Generic (PLEG): container finished" podID="7834721e-c7e2-4c8e-9d6b-96ab1ce496e5" containerID="02065ee61f4d5ab13d73167ee19153b1554f92da0e765c28cc7b72c8c6cccf16" exitCode=0 Oct 30 22:25:05.977042 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:05.976591 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" event={"ID":"7834721e-c7e2-4c8e-9d6b-96ab1ce496e5","Type":"ContainerDied","Data":"02065ee61f4d5ab13d73167ee19153b1554f92da0e765c28cc7b72c8c6cccf16"} Oct 30 22:25:05.978057 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:05.978035 5949 generic.go:356] "Generic (PLEG): container finished" podID="a939103f-a5b6-455e-a716-eaab00eee319" containerID="73d555f6a71d8fbb3068a9f12c16f47df3aeaa8883a21097130603346a929903" exitCode=0 Oct 30 22:25:05.978142 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:05.978092 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" event={"ID":"a939103f-a5b6-455e-a716-eaab00eee319","Type":"ContainerDied","Data":"73d555f6a71d8fbb3068a9f12c16f47df3aeaa8883a21097130603346a929903"} Oct 30 22:25:06.983929 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:06.983893 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" event={"ID":"7834721e-c7e2-4c8e-9d6b-96ab1ce496e5","Type":"ContainerStarted","Data":"92ab430cb7ccecfca6a3322efe9a90ad17457ede02c4f488bc6e1612fc6a2d0a"} Oct 30 22:25:06.984504 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:06.983964 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" event={"ID":"7834721e-c7e2-4c8e-9d6b-96ab1ce496e5","Type":"ContainerStarted","Data":"438ffbcd743e996b4313640e93970fe59aeebcb3b815fa39d8e603aafe62b092"} Oct 30 22:25:06.984504 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:06.984187 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" Oct 30 22:25:06.986009 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:06.985987 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" event={"ID":"a939103f-a5b6-455e-a716-eaab00eee319","Type":"ContainerStarted","Data":"595a903f53ed6ec4fce2a6a7fff0d25ed8dc952435e953947db065fd5b1c7963"} Oct 30 22:25:06.986116 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:06.986033 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" event={"ID":"a939103f-a5b6-455e-a716-eaab00eee319","Type":"ContainerStarted","Data":"a364727db419fc2942daf0ae0729080bfc85ceda0471c023899815d9078c73cc"} Oct 30 22:25:06.986244 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:06.986213 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" Oct 30 22:25:06.986287 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:06.986272 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" Oct 30 22:25:06.987656 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:06.987633 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" podUID="a939103f-a5b6-455e-a716-eaab00eee319" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused" Oct 30 22:25:07.038212 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:07.038166 5949 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" podStartSLOduration=3.075165933 podStartE2EDuration="7.038153357s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.962987424s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:25:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:25:07.037783564 +0000 UTC m=+995.001592133" watchObservedRunningTime="2026-10-30 22:25:07.038153357 +0000 UTC m=+995.001961927" Oct 30 22:25:07.082694 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:07.082650 5949 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" podStartSLOduration=3.012291442 podStartE2EDuration="7.08263805s" totalImagesPullingTime="0s" totalInitContainerRuntime="4.070346608s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:25:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:25:07.082490397 +0000 UTC m=+995.046298968" watchObservedRunningTime="2026-10-30 22:25:07.08263805 +0000 UTC m=+995.046446617" Oct 30 22:25:07.990504 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:07.990455 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" podUID="a939103f-a5b6-455e-a716-eaab00eee319" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused" Oct 30 22:25:07.990997 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:07.990727 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" Oct 30 22:25:07.991838 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:07.991812 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" podUID="7834721e-c7e2-4c8e-9d6b-96ab1ce496e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused" Oct 30 22:25:08.993776 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:08.993737 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" podUID="7834721e-c7e2-4c8e-9d6b-96ab1ce496e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused" Oct 30 22:25:12.995682 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:12.995654 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" Oct 30 22:25:12.996193 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:12.996167 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" podUID="a939103f-a5b6-455e-a716-eaab00eee319" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused" Oct 30 22:25:13.998584 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:13.998514 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" Oct 30 22:25:13.999042 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:13.999018 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" podUID="7834721e-c7e2-4c8e-9d6b-96ab1ce496e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused" Oct 30 22:25:22.996888 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:22.996848 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" podUID="a939103f-a5b6-455e-a716-eaab00eee319" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused" Oct 30 22:25:23.999312 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:23.999272 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" podUID="7834721e-c7e2-4c8e-9d6b-96ab1ce496e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused" Oct 30 22:25:25.336428 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:25:25.336388 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:25:32.996238 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:32.996184 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" podUID="a939103f-a5b6-455e-a716-eaab00eee319" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused" Oct 30 22:25:33.999083 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:33.999047 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" podUID="7834721e-c7e2-4c8e-9d6b-96ab1ce496e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused" Oct 30 22:25:42.996936 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:42.996900 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" podUID="a939103f-a5b6-455e-a716-eaab00eee319" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused" Oct 30 22:25:43.999927 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:43.999883 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" podUID="7834721e-c7e2-4c8e-9d6b-96ab1ce496e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused" Oct 30 22:25:52.997089 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:52.997044 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" podUID="a939103f-a5b6-455e-a716-eaab00eee319" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused" Oct 30 22:25:53.999743 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:25:53.999700 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" podUID="7834721e-c7e2-4c8e-9d6b-96ab1ce496e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused" Oct 30 22:25:55.335803 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:25:55.335765 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:26:02.996949 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:02.996907 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" podUID="a939103f-a5b6-455e-a716-eaab00eee319" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused" Oct 30 22:26:03.999720 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:03.999680 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" podUID="7834721e-c7e2-4c8e-9d6b-96ab1ce496e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused" Oct 30 22:26:12.996536 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:12.996494 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" podUID="a939103f-a5b6-455e-a716-eaab00eee319" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused" Oct 30 22:26:13.999036 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:13.998992 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" podUID="7834721e-c7e2-4c8e-9d6b-96ab1ce496e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused" Oct 30 22:26:22.996405 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:22.996374 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" Oct 30 22:26:24.000055 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:24.000020 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" Oct 30 22:26:24.447158 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:24.447130 5949 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6"] Oct 30 22:26:24.447399 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:24.447357 5949 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" podUID="a939103f-a5b6-455e-a716-eaab00eee319" containerName="kserve-container" containerID="cri-o://a364727db419fc2942daf0ae0729080bfc85ceda0471c023899815d9078c73cc" gracePeriod=30 Oct 30 22:26:24.447471 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:24.447424 5949 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" podUID="a939103f-a5b6-455e-a716-eaab00eee319" containerName="kube-rbac-proxy" containerID="cri-o://595a903f53ed6ec4fce2a6a7fff0d25ed8dc952435e953947db065fd5b1c7963" gracePeriod=30 Oct 30 22:26:25.285917 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:25.285883 5949 generic.go:356] "Generic (PLEG): container finished" podID="a939103f-a5b6-455e-a716-eaab00eee319" containerID="595a903f53ed6ec4fce2a6a7fff0d25ed8dc952435e953947db065fd5b1c7963" exitCode=0 Oct 30 22:26:25.286320 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:25.285951 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" event={"ID":"a939103f-a5b6-455e-a716-eaab00eee319","Type":"ContainerDied","Data":"595a903f53ed6ec4fce2a6a7fff0d25ed8dc952435e953947db065fd5b1c7963"} Oct 30 22:26:25.334736 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:26:25.334699 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:26:27.991291 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:27.991252 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" podUID="a939103f-a5b6-455e-a716-eaab00eee319" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.25:8643/healthz\": dial tcp 10.134.0.25:8643: connect: connection refused" Oct 30 22:26:29.098214 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:29.098188 5949 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" Oct 30 22:26:29.180209 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:29.180181 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a939103f-a5b6-455e-a716-eaab00eee319-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a939103f-a5b6-455e-a716-eaab00eee319-kserve-provision-location\") pod \"a939103f-a5b6-455e-a716-eaab00eee319\" (UID: \"a939103f-a5b6-455e-a716-eaab00eee319\") " Oct 30 22:26:29.180399 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:29.180241 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a939103f-a5b6-455e-a716-eaab00eee319-kube-api-access-55z9j\" (UniqueName: \"kubernetes.io/projected/a939103f-a5b6-455e-a716-eaab00eee319-kube-api-access-55z9j\") pod \"a939103f-a5b6-455e-a716-eaab00eee319\" (UID: \"a939103f-a5b6-455e-a716-eaab00eee319\") " Oct 30 22:26:29.180399 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:29.180302 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a939103f-a5b6-455e-a716-eaab00eee319-proxy-tls\" (UniqueName: \"kubernetes.io/secret/a939103f-a5b6-455e-a716-eaab00eee319-proxy-tls\") pod \"a939103f-a5b6-455e-a716-eaab00eee319\" (UID: \"a939103f-a5b6-455e-a716-eaab00eee319\") " Oct 30 22:26:29.180399 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:29.180323 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a939103f-a5b6-455e-a716-eaab00eee319-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a939103f-a5b6-455e-a716-eaab00eee319-cabundle-cert\") pod \"a939103f-a5b6-455e-a716-eaab00eee319\" (UID: \"a939103f-a5b6-455e-a716-eaab00eee319\") " Oct 30 22:26:29.180399 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:29.180343 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a939103f-a5b6-455e-a716-eaab00eee319-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a939103f-a5b6-455e-a716-eaab00eee319-kube-rbac-proxy-sar-config\") pod \"a939103f-a5b6-455e-a716-eaab00eee319\" (UID: \"a939103f-a5b6-455e-a716-eaab00eee319\") " Oct 30 22:26:29.180656 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:29.180619 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a939103f-a5b6-455e-a716-eaab00eee319-kserve-provision-location" pod "a939103f-a5b6-455e-a716-eaab00eee319" (UID: "a939103f-a5b6-455e-a716-eaab00eee319"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 22:26:29.180738 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:29.180714 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a939103f-a5b6-455e-a716-eaab00eee319-cabundle-cert" pod "a939103f-a5b6-455e-a716-eaab00eee319" (UID: "a939103f-a5b6-455e-a716-eaab00eee319"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:26:29.180796 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:29.180776 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a939103f-a5b6-455e-a716-eaab00eee319-kube-rbac-proxy-sar-config" pod "a939103f-a5b6-455e-a716-eaab00eee319" (UID: "a939103f-a5b6-455e-a716-eaab00eee319"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:26:29.182367 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:29.182346 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a939103f-a5b6-455e-a716-eaab00eee319-kube-api-access-55z9j" pod "a939103f-a5b6-455e-a716-eaab00eee319" (UID: "a939103f-a5b6-455e-a716-eaab00eee319"). InnerVolumeSpecName "kube-api-access-55z9j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 22:26:29.182466 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:29.182429 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a939103f-a5b6-455e-a716-eaab00eee319-proxy-tls" pod "a939103f-a5b6-455e-a716-eaab00eee319" (UID: "a939103f-a5b6-455e-a716-eaab00eee319"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:26:29.280979 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:29.280913 5949 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a939103f-a5b6-455e-a716-eaab00eee319-kserve-provision-location\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:26:29.280979 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:29.280942 5949 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55z9j\" (UniqueName: \"kubernetes.io/projected/a939103f-a5b6-455e-a716-eaab00eee319-kube-api-access-55z9j\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:26:29.280979 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:29.280958 5949 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a939103f-a5b6-455e-a716-eaab00eee319-proxy-tls\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:26:29.280979 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:29.280974 5949 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a939103f-a5b6-455e-a716-eaab00eee319-cabundle-cert\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:26:29.281272 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:29.280987 5949 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a939103f-a5b6-455e-a716-eaab00eee319-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:26:29.302456 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:29.302431 5949 generic.go:356] "Generic (PLEG): container finished" podID="a939103f-a5b6-455e-a716-eaab00eee319" containerID="a364727db419fc2942daf0ae0729080bfc85ceda0471c023899815d9078c73cc" exitCode=0 Oct 30 22:26:29.302564 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:29.302491 5949 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" Oct 30 22:26:29.302564 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:29.302513 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" event={"ID":"a939103f-a5b6-455e-a716-eaab00eee319","Type":"ContainerDied","Data":"a364727db419fc2942daf0ae0729080bfc85ceda0471c023899815d9078c73cc"} Oct 30 22:26:29.302645 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:29.302576 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6" event={"ID":"a939103f-a5b6-455e-a716-eaab00eee319","Type":"ContainerDied","Data":"260afd5ccabba36dcced75f5fb4f04b2c97ec721962334193316eb309c2611b2"} Oct 30 22:26:29.302645 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:29.302605 5949 scope.go:122] "RemoveContainer" containerID="595a903f53ed6ec4fce2a6a7fff0d25ed8dc952435e953947db065fd5b1c7963" Oct 30 22:26:29.311005 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:29.310990 5949 scope.go:122] "RemoveContainer" containerID="a364727db419fc2942daf0ae0729080bfc85ceda0471c023899815d9078c73cc" Oct 30 22:26:29.318056 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:29.318041 5949 scope.go:122] "RemoveContainer" containerID="73d555f6a71d8fbb3068a9f12c16f47df3aeaa8883a21097130603346a929903" Oct 30 22:26:29.324730 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:29.324716 5949 scope.go:122] "RemoveContainer" containerID="595a903f53ed6ec4fce2a6a7fff0d25ed8dc952435e953947db065fd5b1c7963" Oct 30 22:26:29.324978 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:26:29.324959 5949 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"595a903f53ed6ec4fce2a6a7fff0d25ed8dc952435e953947db065fd5b1c7963\": container with ID starting with 595a903f53ed6ec4fce2a6a7fff0d25ed8dc952435e953947db065fd5b1c7963 not found: ID does not exist" containerID="595a903f53ed6ec4fce2a6a7fff0d25ed8dc952435e953947db065fd5b1c7963" Oct 30 22:26:29.325048 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:29.324989 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"595a903f53ed6ec4fce2a6a7fff0d25ed8dc952435e953947db065fd5b1c7963"} err="failed to get container status \"595a903f53ed6ec4fce2a6a7fff0d25ed8dc952435e953947db065fd5b1c7963\": rpc error: code = NotFound desc = could not find container \"595a903f53ed6ec4fce2a6a7fff0d25ed8dc952435e953947db065fd5b1c7963\": container with ID starting with 595a903f53ed6ec4fce2a6a7fff0d25ed8dc952435e953947db065fd5b1c7963 not found: ID does not exist" Oct 30 22:26:29.325048 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:29.325016 5949 scope.go:122] "RemoveContainer" containerID="a364727db419fc2942daf0ae0729080bfc85ceda0471c023899815d9078c73cc" Oct 30 22:26:29.325293 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:26:29.325276 5949 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a364727db419fc2942daf0ae0729080bfc85ceda0471c023899815d9078c73cc\": container with ID starting with a364727db419fc2942daf0ae0729080bfc85ceda0471c023899815d9078c73cc not found: ID does not exist" containerID="a364727db419fc2942daf0ae0729080bfc85ceda0471c023899815d9078c73cc" Oct 30 22:26:29.325341 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:29.325299 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a364727db419fc2942daf0ae0729080bfc85ceda0471c023899815d9078c73cc"} err="failed to get container status \"a364727db419fc2942daf0ae0729080bfc85ceda0471c023899815d9078c73cc\": rpc error: code = NotFound desc = could not find container \"a364727db419fc2942daf0ae0729080bfc85ceda0471c023899815d9078c73cc\": container with ID starting with a364727db419fc2942daf0ae0729080bfc85ceda0471c023899815d9078c73cc not found: ID does not exist" Oct 30 22:26:29.325341 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:29.325315 5949 scope.go:122] "RemoveContainer" containerID="73d555f6a71d8fbb3068a9f12c16f47df3aeaa8883a21097130603346a929903" Oct 30 22:26:29.325521 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:26:29.325503 5949 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73d555f6a71d8fbb3068a9f12c16f47df3aeaa8883a21097130603346a929903\": container with ID starting with 73d555f6a71d8fbb3068a9f12c16f47df3aeaa8883a21097130603346a929903 not found: ID does not exist" containerID="73d555f6a71d8fbb3068a9f12c16f47df3aeaa8883a21097130603346a929903" Oct 30 22:26:29.325568 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:29.325529 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73d555f6a71d8fbb3068a9f12c16f47df3aeaa8883a21097130603346a929903"} err="failed to get container status \"73d555f6a71d8fbb3068a9f12c16f47df3aeaa8883a21097130603346a929903\": rpc error: code = NotFound desc = could not find container \"73d555f6a71d8fbb3068a9f12c16f47df3aeaa8883a21097130603346a929903\": container with ID starting with 73d555f6a71d8fbb3068a9f12c16f47df3aeaa8883a21097130603346a929903 not found: ID does not exist" Oct 30 22:26:29.352525 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:29.352506 5949 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6"] Oct 30 22:26:29.358406 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:29.358386 5949 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-stop-predictor-6bd757b497-478x6"] Oct 30 22:26:30.159190 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:30.159158 5949 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a939103f-a5b6-455e-a716-eaab00eee319" path="/var/lib/kubelet/pods/a939103f-a5b6-455e-a716-eaab00eee319/volumes" Oct 30 22:26:30.377568 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:30.377539 5949 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5"] Oct 30 22:26:30.377830 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:30.377805 5949 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" podUID="7834721e-c7e2-4c8e-9d6b-96ab1ce496e5" containerName="kserve-container" containerID="cri-o://438ffbcd743e996b4313640e93970fe59aeebcb3b815fa39d8e603aafe62b092" gracePeriod=30 Oct 30 22:26:30.377920 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:30.377844 5949 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" podUID="7834721e-c7e2-4c8e-9d6b-96ab1ce496e5" containerName="kube-rbac-proxy" containerID="cri-o://92ab430cb7ccecfca6a3322efe9a90ad17457ede02c4f488bc6e1612fc6a2d0a" gracePeriod=30 Oct 30 22:26:31.313906 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:31.313870 5949 generic.go:356] "Generic (PLEG): container finished" podID="7834721e-c7e2-4c8e-9d6b-96ab1ce496e5" containerID="92ab430cb7ccecfca6a3322efe9a90ad17457ede02c4f488bc6e1612fc6a2d0a" exitCode=0 Oct 30 22:26:31.314297 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:31.313945 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" event={"ID":"7834721e-c7e2-4c8e-9d6b-96ab1ce496e5","Type":"ContainerDied","Data":"92ab430cb7ccecfca6a3322efe9a90ad17457ede02c4f488bc6e1612fc6a2d0a"} Oct 30 22:26:33.994963 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:33.994923 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" podUID="7834721e-c7e2-4c8e-9d6b-96ab1ce496e5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.24:8643/healthz\": dial tcp 10.134.0.24:8643: connect: connection refused" Oct 30 22:26:33.999423 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:33.999402 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" podUID="7834721e-c7e2-4c8e-9d6b-96ab1ce496e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused" Oct 30 22:26:35.117271 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:35.117246 5949 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" Oct 30 22:26:35.227740 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:35.227665 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7834721e-c7e2-4c8e-9d6b-96ab1ce496e5-proxy-tls\" (UniqueName: \"kubernetes.io/secret/7834721e-c7e2-4c8e-9d6b-96ab1ce496e5-proxy-tls\") pod \"7834721e-c7e2-4c8e-9d6b-96ab1ce496e5\" (UID: \"7834721e-c7e2-4c8e-9d6b-96ab1ce496e5\") " Oct 30 22:26:35.227740 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:35.227719 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7834721e-c7e2-4c8e-9d6b-96ab1ce496e5-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7834721e-c7e2-4c8e-9d6b-96ab1ce496e5-kserve-provision-location\") pod \"7834721e-c7e2-4c8e-9d6b-96ab1ce496e5\" (UID: \"7834721e-c7e2-4c8e-9d6b-96ab1ce496e5\") " Oct 30 22:26:35.227966 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:35.227750 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7834721e-c7e2-4c8e-9d6b-96ab1ce496e5-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7834721e-c7e2-4c8e-9d6b-96ab1ce496e5-cabundle-cert\") pod \"7834721e-c7e2-4c8e-9d6b-96ab1ce496e5\" (UID: \"7834721e-c7e2-4c8e-9d6b-96ab1ce496e5\") " Oct 30 22:26:35.227966 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:35.227769 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7834721e-c7e2-4c8e-9d6b-96ab1ce496e5-kube-api-access-2rrfm\" (UniqueName: \"kubernetes.io/projected/7834721e-c7e2-4c8e-9d6b-96ab1ce496e5-kube-api-access-2rrfm\") pod \"7834721e-c7e2-4c8e-9d6b-96ab1ce496e5\" (UID: \"7834721e-c7e2-4c8e-9d6b-96ab1ce496e5\") " Oct 30 22:26:35.227966 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:35.227863 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7834721e-c7e2-4c8e-9d6b-96ab1ce496e5-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7834721e-c7e2-4c8e-9d6b-96ab1ce496e5-kube-rbac-proxy-sar-config\") pod \"7834721e-c7e2-4c8e-9d6b-96ab1ce496e5\" (UID: \"7834721e-c7e2-4c8e-9d6b-96ab1ce496e5\") " Oct 30 22:26:35.228146 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:35.228114 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7834721e-c7e2-4c8e-9d6b-96ab1ce496e5-kserve-provision-location" pod "7834721e-c7e2-4c8e-9d6b-96ab1ce496e5" (UID: "7834721e-c7e2-4c8e-9d6b-96ab1ce496e5"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 22:26:35.228263 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:35.228212 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7834721e-c7e2-4c8e-9d6b-96ab1ce496e5-cabundle-cert" pod "7834721e-c7e2-4c8e-9d6b-96ab1ce496e5" (UID: "7834721e-c7e2-4c8e-9d6b-96ab1ce496e5"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:26:35.228369 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:35.228345 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7834721e-c7e2-4c8e-9d6b-96ab1ce496e5-kube-rbac-proxy-sar-config" pod "7834721e-c7e2-4c8e-9d6b-96ab1ce496e5" (UID: "7834721e-c7e2-4c8e-9d6b-96ab1ce496e5"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:26:35.229891 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:35.229865 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7834721e-c7e2-4c8e-9d6b-96ab1ce496e5-proxy-tls" pod "7834721e-c7e2-4c8e-9d6b-96ab1ce496e5" (UID: "7834721e-c7e2-4c8e-9d6b-96ab1ce496e5"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:26:35.230072 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:35.230050 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7834721e-c7e2-4c8e-9d6b-96ab1ce496e5-kube-api-access-2rrfm" pod "7834721e-c7e2-4c8e-9d6b-96ab1ce496e5" (UID: "7834721e-c7e2-4c8e-9d6b-96ab1ce496e5"). InnerVolumeSpecName "kube-api-access-2rrfm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 22:26:35.328514 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:35.328484 5949 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7834721e-c7e2-4c8e-9d6b-96ab1ce496e5-cabundle-cert\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:26:35.328610 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:35.328515 5949 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2rrfm\" (UniqueName: \"kubernetes.io/projected/7834721e-c7e2-4c8e-9d6b-96ab1ce496e5-kube-api-access-2rrfm\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:26:35.328610 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:35.328530 5949 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7834721e-c7e2-4c8e-9d6b-96ab1ce496e5-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:26:35.328610 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:35.328543 5949 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7834721e-c7e2-4c8e-9d6b-96ab1ce496e5-proxy-tls\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:26:35.328610 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:35.328554 5949 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7834721e-c7e2-4c8e-9d6b-96ab1ce496e5-kserve-provision-location\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:26:35.330167 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:35.330140 5949 generic.go:356] "Generic (PLEG): container finished" podID="7834721e-c7e2-4c8e-9d6b-96ab1ce496e5" containerID="438ffbcd743e996b4313640e93970fe59aeebcb3b815fa39d8e603aafe62b092" exitCode=0 Oct 30 22:26:35.330264 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:35.330195 5949 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" Oct 30 22:26:35.330264 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:35.330240 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" event={"ID":"7834721e-c7e2-4c8e-9d6b-96ab1ce496e5","Type":"ContainerDied","Data":"438ffbcd743e996b4313640e93970fe59aeebcb3b815fa39d8e603aafe62b092"} Oct 30 22:26:35.330343 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:35.330296 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5" event={"ID":"7834721e-c7e2-4c8e-9d6b-96ab1ce496e5","Type":"ContainerDied","Data":"0d91750d050f471be4880c24d56c22443500e451b0cd5b8c6d4bf1912fa86995"} Oct 30 22:26:35.330343 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:35.330324 5949 scope.go:122] "RemoveContainer" containerID="92ab430cb7ccecfca6a3322efe9a90ad17457ede02c4f488bc6e1612fc6a2d0a" Oct 30 22:26:35.338874 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:35.338859 5949 scope.go:122] "RemoveContainer" containerID="438ffbcd743e996b4313640e93970fe59aeebcb3b815fa39d8e603aafe62b092" Oct 30 22:26:35.346371 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:35.346354 5949 scope.go:122] "RemoveContainer" containerID="02065ee61f4d5ab13d73167ee19153b1554f92da0e765c28cc7b72c8c6cccf16" Oct 30 22:26:35.353196 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:35.353179 5949 scope.go:122] "RemoveContainer" containerID="92ab430cb7ccecfca6a3322efe9a90ad17457ede02c4f488bc6e1612fc6a2d0a" Oct 30 22:26:35.353479 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:26:35.353461 5949 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92ab430cb7ccecfca6a3322efe9a90ad17457ede02c4f488bc6e1612fc6a2d0a\": container with ID starting with 92ab430cb7ccecfca6a3322efe9a90ad17457ede02c4f488bc6e1612fc6a2d0a not found: ID does not exist" containerID="92ab430cb7ccecfca6a3322efe9a90ad17457ede02c4f488bc6e1612fc6a2d0a" Oct 30 22:26:35.353527 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:35.353488 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92ab430cb7ccecfca6a3322efe9a90ad17457ede02c4f488bc6e1612fc6a2d0a"} err="failed to get container status \"92ab430cb7ccecfca6a3322efe9a90ad17457ede02c4f488bc6e1612fc6a2d0a\": rpc error: code = NotFound desc = could not find container \"92ab430cb7ccecfca6a3322efe9a90ad17457ede02c4f488bc6e1612fc6a2d0a\": container with ID starting with 92ab430cb7ccecfca6a3322efe9a90ad17457ede02c4f488bc6e1612fc6a2d0a not found: ID does not exist" Oct 30 22:26:35.353527 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:35.353508 5949 scope.go:122] "RemoveContainer" containerID="438ffbcd743e996b4313640e93970fe59aeebcb3b815fa39d8e603aafe62b092" Oct 30 22:26:35.353745 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:26:35.353728 5949 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"438ffbcd743e996b4313640e93970fe59aeebcb3b815fa39d8e603aafe62b092\": container with ID starting with 438ffbcd743e996b4313640e93970fe59aeebcb3b815fa39d8e603aafe62b092 not found: ID does not exist" containerID="438ffbcd743e996b4313640e93970fe59aeebcb3b815fa39d8e603aafe62b092" Oct 30 22:26:35.353784 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:35.353750 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"438ffbcd743e996b4313640e93970fe59aeebcb3b815fa39d8e603aafe62b092"} err="failed to get container status \"438ffbcd743e996b4313640e93970fe59aeebcb3b815fa39d8e603aafe62b092\": rpc error: code = NotFound desc = could not find container \"438ffbcd743e996b4313640e93970fe59aeebcb3b815fa39d8e603aafe62b092\": container with ID starting with 438ffbcd743e996b4313640e93970fe59aeebcb3b815fa39d8e603aafe62b092 not found: ID does not exist" Oct 30 22:26:35.353784 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:35.353764 5949 scope.go:122] "RemoveContainer" containerID="02065ee61f4d5ab13d73167ee19153b1554f92da0e765c28cc7b72c8c6cccf16" Oct 30 22:26:35.353974 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:26:35.353960 5949 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02065ee61f4d5ab13d73167ee19153b1554f92da0e765c28cc7b72c8c6cccf16\": container with ID starting with 02065ee61f4d5ab13d73167ee19153b1554f92da0e765c28cc7b72c8c6cccf16 not found: ID does not exist" containerID="02065ee61f4d5ab13d73167ee19153b1554f92da0e765c28cc7b72c8c6cccf16" Oct 30 22:26:35.354013 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:35.353976 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02065ee61f4d5ab13d73167ee19153b1554f92da0e765c28cc7b72c8c6cccf16"} err="failed to get container status \"02065ee61f4d5ab13d73167ee19153b1554f92da0e765c28cc7b72c8c6cccf16\": rpc error: code = NotFound desc = could not find container \"02065ee61f4d5ab13d73167ee19153b1554f92da0e765c28cc7b72c8c6cccf16\": container with ID starting with 02065ee61f4d5ab13d73167ee19153b1554f92da0e765c28cc7b72c8c6cccf16 not found: ID does not exist" Oct 30 22:26:35.383106 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:35.383082 5949 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5"] Oct 30 22:26:35.390409 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:35.390387 5949 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-stop-v2-predictor-6ff6b7c96-22kw5"] Oct 30 22:26:36.159513 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:26:36.159478 5949 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7834721e-c7e2-4c8e-9d6b-96ab1ce496e5" path="/var/lib/kubelet/pods/7834721e-c7e2-4c8e-9d6b-96ab1ce496e5/volumes" Oct 30 22:26:55.334814 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:26:55.334742 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:27:25.335951 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:27:25.335913 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:27:55.335186 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:27:55.335142 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:28:24.841200 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:24.841167 5949 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-306b77ff42/isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq"] Oct 30 22:28:24.841625 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:24.841529 5949 cpu_manager.go:414] "containerMap: removing container" podUID="a939103f-a5b6-455e-a716-eaab00eee319" containerName="kserve-container" Oct 30 22:28:24.841625 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:24.841541 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a939103f-a5b6-455e-a716-eaab00eee319" containerName="kserve-container" Oct 30 22:28:24.841625 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:24.841553 5949 cpu_manager.go:414] "containerMap: removing container" podUID="7834721e-c7e2-4c8e-9d6b-96ab1ce496e5" containerName="kube-rbac-proxy" Oct 30 22:28:24.841625 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:24.841559 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7834721e-c7e2-4c8e-9d6b-96ab1ce496e5" containerName="kube-rbac-proxy" Oct 30 22:28:24.841625 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:24.841570 5949 cpu_manager.go:414] "containerMap: removing container" podUID="a939103f-a5b6-455e-a716-eaab00eee319" containerName="storage-initializer" Oct 30 22:28:24.841625 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:24.841575 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a939103f-a5b6-455e-a716-eaab00eee319" containerName="storage-initializer" Oct 30 22:28:24.841625 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:24.841580 5949 cpu_manager.go:414] "containerMap: removing container" podUID="a939103f-a5b6-455e-a716-eaab00eee319" containerName="kube-rbac-proxy" Oct 30 22:28:24.841625 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:24.841585 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a939103f-a5b6-455e-a716-eaab00eee319" containerName="kube-rbac-proxy" Oct 30 22:28:24.841625 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:24.841591 5949 cpu_manager.go:414] "containerMap: removing container" podUID="7834721e-c7e2-4c8e-9d6b-96ab1ce496e5" containerName="storage-initializer" Oct 30 22:28:24.841625 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:24.841595 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7834721e-c7e2-4c8e-9d6b-96ab1ce496e5" containerName="storage-initializer" Oct 30 22:28:24.841625 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:24.841609 5949 cpu_manager.go:414] "containerMap: removing container" podUID="7834721e-c7e2-4c8e-9d6b-96ab1ce496e5" containerName="kserve-container" Oct 30 22:28:24.841625 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:24.841613 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7834721e-c7e2-4c8e-9d6b-96ab1ce496e5" containerName="kserve-container" Oct 30 22:28:24.841972 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:24.841665 5949 memory_manager.go:365] "RemoveStaleState removing state" podUID="7834721e-c7e2-4c8e-9d6b-96ab1ce496e5" containerName="kserve-container" Oct 30 22:28:24.841972 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:24.841677 5949 memory_manager.go:365] "RemoveStaleState removing state" podUID="a939103f-a5b6-455e-a716-eaab00eee319" containerName="kserve-container" Oct 30 22:28:24.841972 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:24.841683 5949 memory_manager.go:365] "RemoveStaleState removing state" podUID="7834721e-c7e2-4c8e-9d6b-96ab1ce496e5" containerName="kube-rbac-proxy" Oct 30 22:28:24.841972 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:24.841690 5949 memory_manager.go:365] "RemoveStaleState removing state" podUID="a939103f-a5b6-455e-a716-eaab00eee319" containerName="kube-rbac-proxy" Oct 30 22:28:24.845101 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:24.845081 5949 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq" Oct 30 22:28:24.871252 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:24.871228 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-306b77ff42\"/\"isvc-secondary-209fd8-kube-rbac-proxy-sar-config\"" Oct 30 22:28:24.871396 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:24.871244 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-306b77ff42\"/\"odh-kserve-custom-ca-bundle\"" Oct 30 22:28:24.871396 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:24.871247 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-306b77ff42\"/\"invalid-s3-secret-209fd8\"" Oct 30 22:28:24.871570 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:24.871247 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-306b77ff42\"/\"isvc-secondary-209fd8-predictor-serving-cert\"" Oct 30 22:28:24.871570 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:24.871264 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-306b77ff42\"/\"openshift-service-ca.crt\"" Oct 30 22:28:24.871570 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:24.871437 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-306b77ff42\"/\"invalid-s3-sa-209fd8-dockercfg-cr7zd\"" Oct 30 22:28:24.871570 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:24.871477 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-306b77ff42\"/\"kube-root-ca.crt\"" Oct 30 22:28:24.872073 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:24.872054 5949 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-306b77ff42/isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq"] Oct 30 22:28:24.923938 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:24.923912 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e7d90a63-ddad-4e31-8f8e-6a4365ed509e-proxy-tls\") pod \"isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq\" (UID: \"e7d90a63-ddad-4e31-8f8e-6a4365ed509e\") " pod="e2e-gw0-306b77ff42/isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq" Oct 30 22:28:24.924053 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:24.923952 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e7d90a63-ddad-4e31-8f8e-6a4365ed509e-kube-rbac-proxy-sar-config\") pod \"isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq\" (UID: \"e7d90a63-ddad-4e31-8f8e-6a4365ed509e\") " pod="e2e-gw0-306b77ff42/isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq" Oct 30 22:28:24.924053 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:24.923984 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e7d90a63-ddad-4e31-8f8e-6a4365ed509e-kserve-provision-location\") pod \"isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq\" (UID: \"e7d90a63-ddad-4e31-8f8e-6a4365ed509e\") " pod="e2e-gw0-306b77ff42/isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq" Oct 30 22:28:24.924125 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:24.924047 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e7d90a63-ddad-4e31-8f8e-6a4365ed509e-cabundle-cert\") pod \"isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq\" (UID: \"e7d90a63-ddad-4e31-8f8e-6a4365ed509e\") " pod="e2e-gw0-306b77ff42/isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq" Oct 30 22:28:24.924125 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:24.924104 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7w96\" (UniqueName: \"kubernetes.io/projected/e7d90a63-ddad-4e31-8f8e-6a4365ed509e-kube-api-access-n7w96\") pod \"isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq\" (UID: \"e7d90a63-ddad-4e31-8f8e-6a4365ed509e\") " pod="e2e-gw0-306b77ff42/isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq" Oct 30 22:28:25.025143 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:25.025112 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e7d90a63-ddad-4e31-8f8e-6a4365ed509e-proxy-tls\") pod \"isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq\" (UID: \"e7d90a63-ddad-4e31-8f8e-6a4365ed509e\") " pod="e2e-gw0-306b77ff42/isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq" Oct 30 22:28:25.025287 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:25.025155 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e7d90a63-ddad-4e31-8f8e-6a4365ed509e-kube-rbac-proxy-sar-config\") pod \"isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq\" (UID: \"e7d90a63-ddad-4e31-8f8e-6a4365ed509e\") " pod="e2e-gw0-306b77ff42/isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq" Oct 30 22:28:25.025287 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:25.025206 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e7d90a63-ddad-4e31-8f8e-6a4365ed509e-kserve-provision-location\") pod \"isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq\" (UID: \"e7d90a63-ddad-4e31-8f8e-6a4365ed509e\") " pod="e2e-gw0-306b77ff42/isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq" Oct 30 22:28:25.025287 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:25.025261 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e7d90a63-ddad-4e31-8f8e-6a4365ed509e-cabundle-cert\") pod \"isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq\" (UID: \"e7d90a63-ddad-4e31-8f8e-6a4365ed509e\") " pod="e2e-gw0-306b77ff42/isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq" Oct 30 22:28:25.025412 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:25.025316 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7w96\" (UniqueName: \"kubernetes.io/projected/e7d90a63-ddad-4e31-8f8e-6a4365ed509e-kube-api-access-n7w96\") pod \"isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq\" (UID: \"e7d90a63-ddad-4e31-8f8e-6a4365ed509e\") " pod="e2e-gw0-306b77ff42/isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq" Oct 30 22:28:25.025737 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:25.025704 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e7d90a63-ddad-4e31-8f8e-6a4365ed509e-kserve-provision-location\") pod \"isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq\" (UID: \"e7d90a63-ddad-4e31-8f8e-6a4365ed509e\") " pod="e2e-gw0-306b77ff42/isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq" Oct 30 22:28:25.025916 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:25.025894 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e7d90a63-ddad-4e31-8f8e-6a4365ed509e-kube-rbac-proxy-sar-config\") pod \"isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq\" (UID: \"e7d90a63-ddad-4e31-8f8e-6a4365ed509e\") " pod="e2e-gw0-306b77ff42/isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq" Oct 30 22:28:25.025965 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:25.025928 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e7d90a63-ddad-4e31-8f8e-6a4365ed509e-cabundle-cert\") pod \"isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq\" (UID: \"e7d90a63-ddad-4e31-8f8e-6a4365ed509e\") " pod="e2e-gw0-306b77ff42/isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq" Oct 30 22:28:25.027631 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:25.027610 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e7d90a63-ddad-4e31-8f8e-6a4365ed509e-proxy-tls\") pod \"isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq\" (UID: \"e7d90a63-ddad-4e31-8f8e-6a4365ed509e\") " pod="e2e-gw0-306b77ff42/isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq" Oct 30 22:28:25.047972 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:25.047949 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7w96\" (UniqueName: \"kubernetes.io/projected/e7d90a63-ddad-4e31-8f8e-6a4365ed509e-kube-api-access-n7w96\") pod \"isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq\" (UID: \"e7d90a63-ddad-4e31-8f8e-6a4365ed509e\") " pod="e2e-gw0-306b77ff42/isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq" Oct 30 22:28:25.161437 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:25.161408 5949 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq" Oct 30 22:28:25.335302 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:28:25.335267 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:28:25.460634 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:25.460563 5949 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-306b77ff42/isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq"] Oct 30 22:28:25.464008 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:28:25.463978 5949 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7d90a63_ddad_4e31_8f8e_6a4365ed509e.slice/crio-d17b5a4283e7402672131acb59c223e0d5583c3a5209a7630c22e091983c594a WatchSource:0}: Error finding container d17b5a4283e7402672131acb59c223e0d5583c3a5209a7630c22e091983c594a: Status 404 returned error can't find the container with id d17b5a4283e7402672131acb59c223e0d5583c3a5209a7630c22e091983c594a Oct 30 22:28:25.740262 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:25.740168 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq" event={"ID":"e7d90a63-ddad-4e31-8f8e-6a4365ed509e","Type":"ContainerStarted","Data":"6e438326b2f2f8b097c15951321af12bed52b2ca7186080e679a0a1f8eb542f0"} Oct 30 22:28:25.740262 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:25.740246 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq" event={"ID":"e7d90a63-ddad-4e31-8f8e-6a4365ed509e","Type":"ContainerStarted","Data":"d17b5a4283e7402672131acb59c223e0d5583c3a5209a7630c22e091983c594a"} Oct 30 22:28:29.756150 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:29.756125 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-306b77ff42_isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq_e7d90a63-ddad-4e31-8f8e-6a4365ed509e/storage-initializer/0.log" Oct 30 22:28:29.756529 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:29.756161 5949 generic.go:356] "Generic (PLEG): container finished" podID="e7d90a63-ddad-4e31-8f8e-6a4365ed509e" containerID="6e438326b2f2f8b097c15951321af12bed52b2ca7186080e679a0a1f8eb542f0" exitCode=1 Oct 30 22:28:29.756529 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:29.756186 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq" event={"ID":"e7d90a63-ddad-4e31-8f8e-6a4365ed509e","Type":"ContainerDied","Data":"6e438326b2f2f8b097c15951321af12bed52b2ca7186080e679a0a1f8eb542f0"} Oct 30 22:28:30.762034 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:30.762006 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-306b77ff42_isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq_e7d90a63-ddad-4e31-8f8e-6a4365ed509e/storage-initializer/0.log" Oct 30 22:28:30.762477 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:30.762058 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq" event={"ID":"e7d90a63-ddad-4e31-8f8e-6a4365ed509e","Type":"ContainerStarted","Data":"2163e48cd4eac72b0cc8d61b6666fdad78e0326a9efbaa76825a2bedc60c7a49"} Oct 30 22:28:32.250266 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:32.250238 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-306b77ff42_isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq_e7d90a63-ddad-4e31-8f8e-6a4365ed509e/storage-initializer/0.log" Oct 30 22:28:32.253949 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:32.253922 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-306b77ff42_isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq_e7d90a63-ddad-4e31-8f8e-6a4365ed509e/storage-initializer/0.log" Oct 30 22:28:32.268550 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:32.268526 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ql6lh_074eeae7-932c-4642-8fea-671898135ab1/ovn-acl-logging/0.log" Oct 30 22:28:32.274491 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:32.274474 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ql6lh_074eeae7-932c-4642-8fea-671898135ab1/ovn-acl-logging/0.log" Oct 30 22:28:33.774131 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:33.774101 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-306b77ff42_isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq_e7d90a63-ddad-4e31-8f8e-6a4365ed509e/storage-initializer/1.log" Oct 30 22:28:33.774542 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:33.774479 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-306b77ff42_isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq_e7d90a63-ddad-4e31-8f8e-6a4365ed509e/storage-initializer/0.log" Oct 30 22:28:33.774542 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:33.774511 5949 generic.go:356] "Generic (PLEG): container finished" podID="e7d90a63-ddad-4e31-8f8e-6a4365ed509e" containerID="2163e48cd4eac72b0cc8d61b6666fdad78e0326a9efbaa76825a2bedc60c7a49" exitCode=1 Oct 30 22:28:33.774627 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:33.774591 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq" event={"ID":"e7d90a63-ddad-4e31-8f8e-6a4365ed509e","Type":"ContainerDied","Data":"2163e48cd4eac72b0cc8d61b6666fdad78e0326a9efbaa76825a2bedc60c7a49"} Oct 30 22:28:33.774695 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:33.774680 5949 scope.go:122] "RemoveContainer" containerID="6e438326b2f2f8b097c15951321af12bed52b2ca7186080e679a0a1f8eb542f0" Oct 30 22:28:33.775038 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:33.775015 5949 scope.go:122] "RemoveContainer" containerID="6e438326b2f2f8b097c15951321af12bed52b2ca7186080e679a0a1f8eb542f0" Oct 30 22:28:33.785418 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:28:33.785390 5949 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"storage-initializer\" with CrashLoopBackOff: \"back-off 10s restarting failed container=storage-initializer pod=isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq_e2e-gw0-306b77ff42(e7d90a63-ddad-4e31-8f8e-6a4365ed509e)\"" pod="e2e-gw0-306b77ff42/isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq" podUID="e7d90a63-ddad-4e31-8f8e-6a4365ed509e" Oct 30 22:28:33.785596 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:28:33.785571 5949 log.go:32] "RemoveContainer from runtime service failed" err="rpc error: code = Unknown desc = failed to delete container k8s_storage-initializer_isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq_e2e-gw0-306b77ff42_e7d90a63-ddad-4e31-8f8e-6a4365ed509e_0 in pod sandbox d17b5a4283e7402672131acb59c223e0d5583c3a5209a7630c22e091983c594a from index: no such id: '6e438326b2f2f8b097c15951321af12bed52b2ca7186080e679a0a1f8eb542f0'" containerID="6e438326b2f2f8b097c15951321af12bed52b2ca7186080e679a0a1f8eb542f0" Oct 30 22:28:33.785678 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:33.785608 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e438326b2f2f8b097c15951321af12bed52b2ca7186080e679a0a1f8eb542f0"} err="rpc error: code = Unknown desc = failed to delete container k8s_storage-initializer_isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq_e2e-gw0-306b77ff42_e7d90a63-ddad-4e31-8f8e-6a4365ed509e_0 in pod sandbox d17b5a4283e7402672131acb59c223e0d5583c3a5209a7630c22e091983c594a from index: no such id: '6e438326b2f2f8b097c15951321af12bed52b2ca7186080e679a0a1f8eb542f0'" Oct 30 22:28:34.779417 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:34.779390 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-306b77ff42_isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq_e7d90a63-ddad-4e31-8f8e-6a4365ed509e/storage-initializer/1.log" Oct 30 22:28:40.895678 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:40.895648 5949 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-306b77ff42/isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq"] Oct 30 22:28:41.026167 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:41.026145 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-306b77ff42_isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq_e7d90a63-ddad-4e31-8f8e-6a4365ed509e/storage-initializer/1.log" Oct 30 22:28:41.029938 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:41.029921 5949 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq" Oct 30 22:28:41.062164 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:41.062138 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e7d90a63-ddad-4e31-8f8e-6a4365ed509e-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e7d90a63-ddad-4e31-8f8e-6a4365ed509e-cabundle-cert\") pod \"e7d90a63-ddad-4e31-8f8e-6a4365ed509e\" (UID: \"e7d90a63-ddad-4e31-8f8e-6a4365ed509e\") " Oct 30 22:28:41.062266 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:41.062244 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e7d90a63-ddad-4e31-8f8e-6a4365ed509e-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e7d90a63-ddad-4e31-8f8e-6a4365ed509e-kserve-provision-location\") pod \"e7d90a63-ddad-4e31-8f8e-6a4365ed509e\" (UID: \"e7d90a63-ddad-4e31-8f8e-6a4365ed509e\") " Oct 30 22:28:41.062320 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:41.062275 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e7d90a63-ddad-4e31-8f8e-6a4365ed509e-kube-api-access-n7w96\" (UniqueName: \"kubernetes.io/projected/e7d90a63-ddad-4e31-8f8e-6a4365ed509e-kube-api-access-n7w96\") pod \"e7d90a63-ddad-4e31-8f8e-6a4365ed509e\" (UID: \"e7d90a63-ddad-4e31-8f8e-6a4365ed509e\") " Oct 30 22:28:41.062320 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:41.062299 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e7d90a63-ddad-4e31-8f8e-6a4365ed509e-proxy-tls\" (UniqueName: \"kubernetes.io/secret/e7d90a63-ddad-4e31-8f8e-6a4365ed509e-proxy-tls\") pod \"e7d90a63-ddad-4e31-8f8e-6a4365ed509e\" (UID: \"e7d90a63-ddad-4e31-8f8e-6a4365ed509e\") " Oct 30 22:28:41.062419 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:41.062342 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e7d90a63-ddad-4e31-8f8e-6a4365ed509e-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e7d90a63-ddad-4e31-8f8e-6a4365ed509e-kube-rbac-proxy-sar-config\") pod \"e7d90a63-ddad-4e31-8f8e-6a4365ed509e\" (UID: \"e7d90a63-ddad-4e31-8f8e-6a4365ed509e\") " Oct 30 22:28:41.062475 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:41.062419 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7d90a63-ddad-4e31-8f8e-6a4365ed509e-kserve-provision-location" pod "e7d90a63-ddad-4e31-8f8e-6a4365ed509e" (UID: "e7d90a63-ddad-4e31-8f8e-6a4365ed509e"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 22:28:41.062524 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:41.062505 5949 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e7d90a63-ddad-4e31-8f8e-6a4365ed509e-kserve-provision-location\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:28:41.062524 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:41.062499 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7d90a63-ddad-4e31-8f8e-6a4365ed509e-cabundle-cert" pod "e7d90a63-ddad-4e31-8f8e-6a4365ed509e" (UID: "e7d90a63-ddad-4e31-8f8e-6a4365ed509e"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:28:41.062691 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:41.062671 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7d90a63-ddad-4e31-8f8e-6a4365ed509e-kube-rbac-proxy-sar-config" pod "e7d90a63-ddad-4e31-8f8e-6a4365ed509e" (UID: "e7d90a63-ddad-4e31-8f8e-6a4365ed509e"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:28:41.064323 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:41.064301 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7d90a63-ddad-4e31-8f8e-6a4365ed509e-proxy-tls" pod "e7d90a63-ddad-4e31-8f8e-6a4365ed509e" (UID: "e7d90a63-ddad-4e31-8f8e-6a4365ed509e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:28:41.064398 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:41.064377 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7d90a63-ddad-4e31-8f8e-6a4365ed509e-kube-api-access-n7w96" pod "e7d90a63-ddad-4e31-8f8e-6a4365ed509e" (UID: "e7d90a63-ddad-4e31-8f8e-6a4365ed509e"). InnerVolumeSpecName "kube-api-access-n7w96". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 22:28:41.162929 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:41.162851 5949 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e7d90a63-ddad-4e31-8f8e-6a4365ed509e-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:28:41.162929 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:41.162878 5949 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e7d90a63-ddad-4e31-8f8e-6a4365ed509e-cabundle-cert\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:28:41.162929 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:41.162888 5949 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n7w96\" (UniqueName: \"kubernetes.io/projected/e7d90a63-ddad-4e31-8f8e-6a4365ed509e-kube-api-access-n7w96\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:28:41.162929 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:41.162896 5949 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e7d90a63-ddad-4e31-8f8e-6a4365ed509e-proxy-tls\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:28:41.806004 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:41.805981 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-306b77ff42_isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq_e7d90a63-ddad-4e31-8f8e-6a4365ed509e/storage-initializer/1.log" Oct 30 22:28:41.806169 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:41.806073 5949 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq" Oct 30 22:28:41.806169 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:41.806082 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq" event={"ID":"e7d90a63-ddad-4e31-8f8e-6a4365ed509e","Type":"ContainerDied","Data":"d17b5a4283e7402672131acb59c223e0d5583c3a5209a7630c22e091983c594a"} Oct 30 22:28:41.806169 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:41.806135 5949 scope.go:122] "RemoveContainer" containerID="2163e48cd4eac72b0cc8d61b6666fdad78e0326a9efbaa76825a2bedc60c7a49" Oct 30 22:28:41.895667 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:41.895635 5949 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-306b77ff42/isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq"] Oct 30 22:28:41.903734 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:41.903710 5949 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-306b77ff42/isvc-secondary-209fd8-predictor-596cbfd64b-mjbwq"] Oct 30 22:28:42.160863 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:42.160833 5949 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e7d90a63-ddad-4e31-8f8e-6a4365ed509e" path="/var/lib/kubelet/pods/e7d90a63-ddad-4e31-8f8e-6a4365ed509e/volumes" Oct 30 22:28:48.722884 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:48.722851 5949 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-306b77ff42/isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7"] Oct 30 22:28:48.723252 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:48.723187 5949 cpu_manager.go:414] "containerMap: removing container" podUID="e7d90a63-ddad-4e31-8f8e-6a4365ed509e" containerName="storage-initializer" Oct 30 22:28:48.723252 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:48.723199 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e7d90a63-ddad-4e31-8f8e-6a4365ed509e" containerName="storage-initializer" Oct 30 22:28:48.723252 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:48.723214 5949 cpu_manager.go:414] "containerMap: removing container" podUID="e7d90a63-ddad-4e31-8f8e-6a4365ed509e" containerName="storage-initializer" Oct 30 22:28:48.723252 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:48.723232 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e7d90a63-ddad-4e31-8f8e-6a4365ed509e" containerName="storage-initializer" Oct 30 22:28:48.723438 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:48.723305 5949 memory_manager.go:365] "RemoveStaleState removing state" podUID="e7d90a63-ddad-4e31-8f8e-6a4365ed509e" containerName="storage-initializer" Oct 30 22:28:48.723438 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:48.723314 5949 memory_manager.go:365] "RemoveStaleState removing state" podUID="e7d90a63-ddad-4e31-8f8e-6a4365ed509e" containerName="storage-initializer" Oct 30 22:28:48.727831 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:48.727811 5949 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7" Oct 30 22:28:48.754856 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:48.754779 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-306b77ff42\"/\"fail-s3-secret-60b815\"" Oct 30 22:28:48.754856 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:48.754801 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-306b77ff42\"/\"odh-kserve-custom-ca-bundle\"" Oct 30 22:28:48.754856 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:48.754813 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-306b77ff42\"/\"fail-s3-sa-60b815-dockercfg-dr4f8\"" Oct 30 22:28:48.755239 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:48.754889 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-306b77ff42\"/\"isvc-init-fail-60b815-kube-rbac-proxy-sar-config\"" Oct 30 22:28:48.755239 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:48.754896 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-306b77ff42\"/\"isvc-init-fail-60b815-predictor-serving-cert\"" Oct 30 22:28:48.755239 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:48.754896 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-306b77ff42\"/\"kube-root-ca.crt\"" Oct 30 22:28:48.755239 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:48.754949 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-306b77ff42\"/\"openshift-service-ca.crt\"" Oct 30 22:28:48.756043 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:48.756024 5949 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-306b77ff42/isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7"] Oct 30 22:28:48.825516 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:48.825487 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5ae2c838-22f5-48ea-ab47-4684783c8833-proxy-tls\") pod \"isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7\" (UID: \"5ae2c838-22f5-48ea-ab47-4684783c8833\") " pod="e2e-gw0-306b77ff42/isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7" Oct 30 22:28:48.825603 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:48.825561 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/5ae2c838-22f5-48ea-ab47-4684783c8833-cabundle-cert\") pod \"isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7\" (UID: \"5ae2c838-22f5-48ea-ab47-4684783c8833\") " pod="e2e-gw0-306b77ff42/isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7" Oct 30 22:28:48.825603 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:48.825597 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5ae2c838-22f5-48ea-ab47-4684783c8833-kube-rbac-proxy-sar-config\") pod \"isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7\" (UID: \"5ae2c838-22f5-48ea-ab47-4684783c8833\") " pod="e2e-gw0-306b77ff42/isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7" Oct 30 22:28:48.825688 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:48.825619 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5ae2c838-22f5-48ea-ab47-4684783c8833-kserve-provision-location\") pod \"isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7\" (UID: \"5ae2c838-22f5-48ea-ab47-4684783c8833\") " pod="e2e-gw0-306b77ff42/isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7" Oct 30 22:28:48.825737 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:48.825691 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4xpj\" (UniqueName: \"kubernetes.io/projected/5ae2c838-22f5-48ea-ab47-4684783c8833-kube-api-access-k4xpj\") pod \"isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7\" (UID: \"5ae2c838-22f5-48ea-ab47-4684783c8833\") " pod="e2e-gw0-306b77ff42/isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7" Oct 30 22:28:48.926171 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:48.926144 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5ae2c838-22f5-48ea-ab47-4684783c8833-kube-rbac-proxy-sar-config\") pod \"isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7\" (UID: \"5ae2c838-22f5-48ea-ab47-4684783c8833\") " pod="e2e-gw0-306b77ff42/isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7" Oct 30 22:28:48.926299 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:48.926176 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5ae2c838-22f5-48ea-ab47-4684783c8833-kserve-provision-location\") pod \"isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7\" (UID: \"5ae2c838-22f5-48ea-ab47-4684783c8833\") " pod="e2e-gw0-306b77ff42/isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7" Oct 30 22:28:48.926364 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:48.926332 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4xpj\" (UniqueName: \"kubernetes.io/projected/5ae2c838-22f5-48ea-ab47-4684783c8833-kube-api-access-k4xpj\") pod \"isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7\" (UID: \"5ae2c838-22f5-48ea-ab47-4684783c8833\") " pod="e2e-gw0-306b77ff42/isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7" Oct 30 22:28:48.926423 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:48.926406 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5ae2c838-22f5-48ea-ab47-4684783c8833-proxy-tls\") pod \"isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7\" (UID: \"5ae2c838-22f5-48ea-ab47-4684783c8833\") " pod="e2e-gw0-306b77ff42/isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7" Oct 30 22:28:48.926482 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:48.926464 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/5ae2c838-22f5-48ea-ab47-4684783c8833-cabundle-cert\") pod \"isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7\" (UID: \"5ae2c838-22f5-48ea-ab47-4684783c8833\") " pod="e2e-gw0-306b77ff42/isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7" Oct 30 22:28:48.926551 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:48.926535 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5ae2c838-22f5-48ea-ab47-4684783c8833-kserve-provision-location\") pod \"isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7\" (UID: \"5ae2c838-22f5-48ea-ab47-4684783c8833\") " pod="e2e-gw0-306b77ff42/isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7" Oct 30 22:28:48.926879 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:48.926857 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5ae2c838-22f5-48ea-ab47-4684783c8833-kube-rbac-proxy-sar-config\") pod \"isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7\" (UID: \"5ae2c838-22f5-48ea-ab47-4684783c8833\") " pod="e2e-gw0-306b77ff42/isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7" Oct 30 22:28:48.927033 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:48.927015 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/5ae2c838-22f5-48ea-ab47-4684783c8833-cabundle-cert\") pod \"isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7\" (UID: \"5ae2c838-22f5-48ea-ab47-4684783c8833\") " pod="e2e-gw0-306b77ff42/isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7" Oct 30 22:28:48.928732 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:48.928715 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5ae2c838-22f5-48ea-ab47-4684783c8833-proxy-tls\") pod \"isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7\" (UID: \"5ae2c838-22f5-48ea-ab47-4684783c8833\") " pod="e2e-gw0-306b77ff42/isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7" Oct 30 22:28:48.949696 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:48.949667 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4xpj\" (UniqueName: \"kubernetes.io/projected/5ae2c838-22f5-48ea-ab47-4684783c8833-kube-api-access-k4xpj\") pod \"isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7\" (UID: \"5ae2c838-22f5-48ea-ab47-4684783c8833\") " pod="e2e-gw0-306b77ff42/isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7" Oct 30 22:28:49.045614 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:49.045552 5949 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7" Oct 30 22:28:49.344898 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:49.344818 5949 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-306b77ff42/isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7"] Oct 30 22:28:49.349262 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:28:49.349230 5949 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ae2c838_22f5_48ea_ab47_4684783c8833.slice/crio-537e3f47786754811c39c8e5fb67fb4d35f926a2233ed215f9e4f847a58fd9df WatchSource:0}: Error finding container 537e3f47786754811c39c8e5fb67fb4d35f926a2233ed215f9e4f847a58fd9df: Status 404 returned error can't find the container with id 537e3f47786754811c39c8e5fb67fb4d35f926a2233ed215f9e4f847a58fd9df Oct 30 22:28:49.839792 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:49.839758 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7" event={"ID":"5ae2c838-22f5-48ea-ab47-4684783c8833","Type":"ContainerStarted","Data":"7bb4b0726968b9728f8b853a34fef38b3150cb9055f7d7d377fb3ef072250d75"} Oct 30 22:28:49.840171 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:49.839817 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7" event={"ID":"5ae2c838-22f5-48ea-ab47-4684783c8833","Type":"ContainerStarted","Data":"537e3f47786754811c39c8e5fb67fb4d35f926a2233ed215f9e4f847a58fd9df"} Oct 30 22:28:52.852745 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:52.852715 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-306b77ff42_isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7_5ae2c838-22f5-48ea-ab47-4684783c8833/storage-initializer/0.log" Oct 30 22:28:52.853165 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:52.852754 5949 generic.go:356] "Generic (PLEG): container finished" podID="5ae2c838-22f5-48ea-ab47-4684783c8833" containerID="7bb4b0726968b9728f8b853a34fef38b3150cb9055f7d7d377fb3ef072250d75" exitCode=1 Oct 30 22:28:52.853165 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:52.852792 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7" event={"ID":"5ae2c838-22f5-48ea-ab47-4684783c8833","Type":"ContainerDied","Data":"7bb4b0726968b9728f8b853a34fef38b3150cb9055f7d7d377fb3ef072250d75"} Oct 30 22:28:53.587312 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:53.587276 5949 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-306b77ff42/isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7"] Oct 30 22:28:53.858456 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:53.858380 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-306b77ff42_isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7_5ae2c838-22f5-48ea-ab47-4684783c8833/storage-initializer/0.log" Oct 30 22:28:53.858456 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:53.858426 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7" event={"ID":"5ae2c838-22f5-48ea-ab47-4684783c8833","Type":"ContainerStarted","Data":"135b4fe83fbdf109c4e5bd8324afd15e88e62d62d7252cc594aaca2635347534"} Oct 30 22:28:53.858929 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:53.858549 5949 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-306b77ff42/isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7" podUID="5ae2c838-22f5-48ea-ab47-4684783c8833" containerName="storage-initializer" containerID="cri-o://135b4fe83fbdf109c4e5bd8324afd15e88e62d62d7252cc594aaca2635347534" gracePeriod=30 Oct 30 22:28:55.335341 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:28:55.335301 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:28:59.410183 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:59.410160 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-306b77ff42_isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7_5ae2c838-22f5-48ea-ab47-4684783c8833/storage-initializer/1.log" Oct 30 22:28:59.410589 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:59.410572 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-306b77ff42_isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7_5ae2c838-22f5-48ea-ab47-4684783c8833/storage-initializer/0.log" Oct 30 22:28:59.413000 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:59.412985 5949 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7" Oct 30 22:28:59.512305 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:59.512230 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5ae2c838-22f5-48ea-ab47-4684783c8833-proxy-tls\" (UniqueName: \"kubernetes.io/secret/5ae2c838-22f5-48ea-ab47-4684783c8833-proxy-tls\") pod \"5ae2c838-22f5-48ea-ab47-4684783c8833\" (UID: \"5ae2c838-22f5-48ea-ab47-4684783c8833\") " Oct 30 22:28:59.512305 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:59.512284 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5ae2c838-22f5-48ea-ab47-4684783c8833-kube-api-access-k4xpj\" (UniqueName: \"kubernetes.io/projected/5ae2c838-22f5-48ea-ab47-4684783c8833-kube-api-access-k4xpj\") pod \"5ae2c838-22f5-48ea-ab47-4684783c8833\" (UID: \"5ae2c838-22f5-48ea-ab47-4684783c8833\") " Oct 30 22:28:59.512532 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:59.512311 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5ae2c838-22f5-48ea-ab47-4684783c8833-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5ae2c838-22f5-48ea-ab47-4684783c8833-kube-rbac-proxy-sar-config\") pod \"5ae2c838-22f5-48ea-ab47-4684783c8833\" (UID: \"5ae2c838-22f5-48ea-ab47-4684783c8833\") " Oct 30 22:28:59.512532 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:59.512332 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5ae2c838-22f5-48ea-ab47-4684783c8833-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5ae2c838-22f5-48ea-ab47-4684783c8833-kserve-provision-location\") pod \"5ae2c838-22f5-48ea-ab47-4684783c8833\" (UID: \"5ae2c838-22f5-48ea-ab47-4684783c8833\") " Oct 30 22:28:59.512532 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:59.512356 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5ae2c838-22f5-48ea-ab47-4684783c8833-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/5ae2c838-22f5-48ea-ab47-4684783c8833-cabundle-cert\") pod \"5ae2c838-22f5-48ea-ab47-4684783c8833\" (UID: \"5ae2c838-22f5-48ea-ab47-4684783c8833\") " Oct 30 22:28:59.512711 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:59.512625 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ae2c838-22f5-48ea-ab47-4684783c8833-kserve-provision-location" pod "5ae2c838-22f5-48ea-ab47-4684783c8833" (UID: "5ae2c838-22f5-48ea-ab47-4684783c8833"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 22:28:59.512765 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:59.512716 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ae2c838-22f5-48ea-ab47-4684783c8833-kube-rbac-proxy-sar-config" pod "5ae2c838-22f5-48ea-ab47-4684783c8833" (UID: "5ae2c838-22f5-48ea-ab47-4684783c8833"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:28:59.512805 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:59.512783 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ae2c838-22f5-48ea-ab47-4684783c8833-cabundle-cert" pod "5ae2c838-22f5-48ea-ab47-4684783c8833" (UID: "5ae2c838-22f5-48ea-ab47-4684783c8833"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:28:59.514248 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:59.514210 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ae2c838-22f5-48ea-ab47-4684783c8833-kube-api-access-k4xpj" pod "5ae2c838-22f5-48ea-ab47-4684783c8833" (UID: "5ae2c838-22f5-48ea-ab47-4684783c8833"). InnerVolumeSpecName "kube-api-access-k4xpj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 22:28:59.514386 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:59.514368 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ae2c838-22f5-48ea-ab47-4684783c8833-proxy-tls" pod "5ae2c838-22f5-48ea-ab47-4684783c8833" (UID: "5ae2c838-22f5-48ea-ab47-4684783c8833"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:28:59.613319 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:59.613289 5949 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5ae2c838-22f5-48ea-ab47-4684783c8833-proxy-tls\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:28:59.613319 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:59.613314 5949 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4xpj\" (UniqueName: \"kubernetes.io/projected/5ae2c838-22f5-48ea-ab47-4684783c8833-kube-api-access-k4xpj\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:28:59.613319 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:59.613326 5949 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5ae2c838-22f5-48ea-ab47-4684783c8833-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:28:59.613551 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:59.613336 5949 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5ae2c838-22f5-48ea-ab47-4684783c8833-kserve-provision-location\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:28:59.613551 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:59.613345 5949 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/5ae2c838-22f5-48ea-ab47-4684783c8833-cabundle-cert\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:28:59.882282 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:59.882196 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-306b77ff42_isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7_5ae2c838-22f5-48ea-ab47-4684783c8833/storage-initializer/1.log" Oct 30 22:28:59.882587 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:59.882573 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-306b77ff42_isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7_5ae2c838-22f5-48ea-ab47-4684783c8833/storage-initializer/0.log" Oct 30 22:28:59.882639 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:59.882606 5949 generic.go:356] "Generic (PLEG): container finished" podID="5ae2c838-22f5-48ea-ab47-4684783c8833" containerID="135b4fe83fbdf109c4e5bd8324afd15e88e62d62d7252cc594aaca2635347534" exitCode=1 Oct 30 22:28:59.882671 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:59.882635 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7" event={"ID":"5ae2c838-22f5-48ea-ab47-4684783c8833","Type":"ContainerDied","Data":"135b4fe83fbdf109c4e5bd8324afd15e88e62d62d7252cc594aaca2635347534"} Oct 30 22:28:59.882705 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:59.882675 5949 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7" Oct 30 22:28:59.882705 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:59.882695 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7" event={"ID":"5ae2c838-22f5-48ea-ab47-4684783c8833","Type":"ContainerDied","Data":"537e3f47786754811c39c8e5fb67fb4d35f926a2233ed215f9e4f847a58fd9df"} Oct 30 22:28:59.882771 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:59.882731 5949 scope.go:122] "RemoveContainer" containerID="135b4fe83fbdf109c4e5bd8324afd15e88e62d62d7252cc594aaca2635347534" Oct 30 22:28:59.891635 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:59.891611 5949 scope.go:122] "RemoveContainer" containerID="7bb4b0726968b9728f8b853a34fef38b3150cb9055f7d7d377fb3ef072250d75" Oct 30 22:28:59.899098 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:59.899079 5949 scope.go:122] "RemoveContainer" containerID="135b4fe83fbdf109c4e5bd8324afd15e88e62d62d7252cc594aaca2635347534" Oct 30 22:28:59.899440 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:28:59.899421 5949 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"135b4fe83fbdf109c4e5bd8324afd15e88e62d62d7252cc594aaca2635347534\": container with ID starting with 135b4fe83fbdf109c4e5bd8324afd15e88e62d62d7252cc594aaca2635347534 not found: ID does not exist" containerID="135b4fe83fbdf109c4e5bd8324afd15e88e62d62d7252cc594aaca2635347534" Oct 30 22:28:59.899503 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:59.899449 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"135b4fe83fbdf109c4e5bd8324afd15e88e62d62d7252cc594aaca2635347534"} err="failed to get container status \"135b4fe83fbdf109c4e5bd8324afd15e88e62d62d7252cc594aaca2635347534\": rpc error: code = NotFound desc = could not find container \"135b4fe83fbdf109c4e5bd8324afd15e88e62d62d7252cc594aaca2635347534\": container with ID starting with 135b4fe83fbdf109c4e5bd8324afd15e88e62d62d7252cc594aaca2635347534 not found: ID does not exist" Oct 30 22:28:59.899503 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:59.899465 5949 scope.go:122] "RemoveContainer" containerID="7bb4b0726968b9728f8b853a34fef38b3150cb9055f7d7d377fb3ef072250d75" Oct 30 22:28:59.899695 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:28:59.899676 5949 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7bb4b0726968b9728f8b853a34fef38b3150cb9055f7d7d377fb3ef072250d75\": container with ID starting with 7bb4b0726968b9728f8b853a34fef38b3150cb9055f7d7d377fb3ef072250d75 not found: ID does not exist" containerID="7bb4b0726968b9728f8b853a34fef38b3150cb9055f7d7d377fb3ef072250d75" Oct 30 22:28:59.899739 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:59.899702 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bb4b0726968b9728f8b853a34fef38b3150cb9055f7d7d377fb3ef072250d75"} err="failed to get container status \"7bb4b0726968b9728f8b853a34fef38b3150cb9055f7d7d377fb3ef072250d75\": rpc error: code = NotFound desc = could not find container \"7bb4b0726968b9728f8b853a34fef38b3150cb9055f7d7d377fb3ef072250d75\": container with ID starting with 7bb4b0726968b9728f8b853a34fef38b3150cb9055f7d7d377fb3ef072250d75 not found: ID does not exist" Oct 30 22:28:59.963287 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:59.963262 5949 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-306b77ff42/isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7"] Oct 30 22:28:59.968428 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:28:59.968400 5949 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-306b77ff42/isvc-init-fail-60b815-predictor-7755cc6c9b-n85b7"] Oct 30 22:29:00.159268 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:00.159244 5949 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5ae2c838-22f5-48ea-ab47-4684783c8833" path="/var/lib/kubelet/pods/5ae2c838-22f5-48ea-ab47-4684783c8833/volumes" Oct 30 22:29:02.537854 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:02.537823 5949 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7"] Oct 30 22:29:02.538270 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:02.538118 5949 cpu_manager.go:414] "containerMap: removing container" podUID="5ae2c838-22f5-48ea-ab47-4684783c8833" containerName="storage-initializer" Oct 30 22:29:02.538270 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:02.538130 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5ae2c838-22f5-48ea-ab47-4684783c8833" containerName="storage-initializer" Oct 30 22:29:02.538270 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:02.538199 5949 memory_manager.go:365] "RemoveStaleState removing state" podUID="5ae2c838-22f5-48ea-ab47-4684783c8833" containerName="storage-initializer" Oct 30 22:29:02.538270 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:02.538207 5949 memory_manager.go:365] "RemoveStaleState removing state" podUID="5ae2c838-22f5-48ea-ab47-4684783c8833" containerName="storage-initializer" Oct 30 22:29:02.538403 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:02.538278 5949 cpu_manager.go:414] "containerMap: removing container" podUID="5ae2c838-22f5-48ea-ab47-4684783c8833" containerName="storage-initializer" Oct 30 22:29:02.538403 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:02.538284 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5ae2c838-22f5-48ea-ab47-4684783c8833" containerName="storage-initializer" Oct 30 22:29:02.542866 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:02.542845 5949 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" Oct 30 22:29:02.561742 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:02.561710 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-306b77ff42\"/\"default-dockercfg-7rhfl\"" Oct 30 22:29:02.564650 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:02.564629 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-306b77ff42\"/\"raw-sklearn-33c9b-predictor-serving-cert\"" Oct 30 22:29:02.564650 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:02.564639 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-306b77ff42\"/\"seaweedfs-s3-creds\"" Oct 30 22:29:02.573559 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:02.573538 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-306b77ff42\"/\"odh-kserve-custom-ca-bundle\"" Oct 30 22:29:02.574326 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:02.574307 5949 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7"] Oct 30 22:29:02.576329 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:02.576313 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-306b77ff42\"/\"openshift-service-ca.crt\"" Oct 30 22:29:02.576393 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:02.576319 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-306b77ff42\"/\"raw-sklearn-33c9b-kube-rbac-proxy-sar-config\"" Oct 30 22:29:02.576393 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:02.576329 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-306b77ff42\"/\"kube-root-ca.crt\"" Oct 30 22:29:02.635597 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:02.635560 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/76c39cd8-6fe1-42f4-8fee-f69513af1ea8-kserve-provision-location\") pod \"raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7\" (UID: \"76c39cd8-6fe1-42f4-8fee-f69513af1ea8\") " pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" Oct 30 22:29:02.635756 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:02.635604 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7766l\" (UniqueName: \"kubernetes.io/projected/76c39cd8-6fe1-42f4-8fee-f69513af1ea8-kube-api-access-7766l\") pod \"raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7\" (UID: \"76c39cd8-6fe1-42f4-8fee-f69513af1ea8\") " pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" Oct 30 22:29:02.635756 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:02.635653 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/76c39cd8-6fe1-42f4-8fee-f69513af1ea8-proxy-tls\") pod \"raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7\" (UID: \"76c39cd8-6fe1-42f4-8fee-f69513af1ea8\") " pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" Oct 30 22:29:02.635839 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:02.635752 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/76c39cd8-6fe1-42f4-8fee-f69513af1ea8-kube-rbac-proxy-sar-config\") pod \"raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7\" (UID: \"76c39cd8-6fe1-42f4-8fee-f69513af1ea8\") " pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" Oct 30 22:29:02.635839 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:02.635775 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/76c39cd8-6fe1-42f4-8fee-f69513af1ea8-cabundle-cert\") pod \"raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7\" (UID: \"76c39cd8-6fe1-42f4-8fee-f69513af1ea8\") " pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" Oct 30 22:29:02.736901 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:02.736865 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/76c39cd8-6fe1-42f4-8fee-f69513af1ea8-proxy-tls\") pod \"raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7\" (UID: \"76c39cd8-6fe1-42f4-8fee-f69513af1ea8\") " pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" Oct 30 22:29:02.736901 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:02.736902 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/76c39cd8-6fe1-42f4-8fee-f69513af1ea8-kube-rbac-proxy-sar-config\") pod \"raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7\" (UID: \"76c39cd8-6fe1-42f4-8fee-f69513af1ea8\") " pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" Oct 30 22:29:02.737117 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:02.736923 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/76c39cd8-6fe1-42f4-8fee-f69513af1ea8-cabundle-cert\") pod \"raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7\" (UID: \"76c39cd8-6fe1-42f4-8fee-f69513af1ea8\") " pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" Oct 30 22:29:02.737117 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:02.736993 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/76c39cd8-6fe1-42f4-8fee-f69513af1ea8-kserve-provision-location\") pod \"raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7\" (UID: \"76c39cd8-6fe1-42f4-8fee-f69513af1ea8\") " pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" Oct 30 22:29:02.737117 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:02.737032 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7766l\" (UniqueName: \"kubernetes.io/projected/76c39cd8-6fe1-42f4-8fee-f69513af1ea8-kube-api-access-7766l\") pod \"raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7\" (UID: \"76c39cd8-6fe1-42f4-8fee-f69513af1ea8\") " pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" Oct 30 22:29:02.737490 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:02.737465 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/76c39cd8-6fe1-42f4-8fee-f69513af1ea8-kserve-provision-location\") pod \"raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7\" (UID: \"76c39cd8-6fe1-42f4-8fee-f69513af1ea8\") " pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" Oct 30 22:29:02.737695 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:02.737677 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/76c39cd8-6fe1-42f4-8fee-f69513af1ea8-kube-rbac-proxy-sar-config\") pod \"raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7\" (UID: \"76c39cd8-6fe1-42f4-8fee-f69513af1ea8\") " pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" Oct 30 22:29:02.737743 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:02.737701 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/76c39cd8-6fe1-42f4-8fee-f69513af1ea8-cabundle-cert\") pod \"raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7\" (UID: \"76c39cd8-6fe1-42f4-8fee-f69513af1ea8\") " pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" Oct 30 22:29:02.739556 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:02.739534 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/76c39cd8-6fe1-42f4-8fee-f69513af1ea8-proxy-tls\") pod \"raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7\" (UID: \"76c39cd8-6fe1-42f4-8fee-f69513af1ea8\") " pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" Oct 30 22:29:02.760380 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:02.760349 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7766l\" (UniqueName: \"kubernetes.io/projected/76c39cd8-6fe1-42f4-8fee-f69513af1ea8-kube-api-access-7766l\") pod \"raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7\" (UID: \"76c39cd8-6fe1-42f4-8fee-f69513af1ea8\") " pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" Oct 30 22:29:02.859679 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:02.859604 5949 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" Oct 30 22:29:03.163265 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:03.163230 5949 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7"] Oct 30 22:29:03.166358 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:29:03.166330 5949 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76c39cd8_6fe1_42f4_8fee_f69513af1ea8.slice/crio-b0389ff0e7cc20a6ea48b635b5dc5d3261c5e2d0485ace78d9bfdc08ab15d1da WatchSource:0}: Error finding container b0389ff0e7cc20a6ea48b635b5dc5d3261c5e2d0485ace78d9bfdc08ab15d1da: Status 404 returned error can't find the container with id b0389ff0e7cc20a6ea48b635b5dc5d3261c5e2d0485ace78d9bfdc08ab15d1da Oct 30 22:29:03.900455 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:03.900421 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" event={"ID":"76c39cd8-6fe1-42f4-8fee-f69513af1ea8","Type":"ContainerStarted","Data":"c00ae369b81d9d3125802a4016559b297d2042fccd115fa661ca5435118f2ce5"} Oct 30 22:29:03.900855 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:03.900479 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" event={"ID":"76c39cd8-6fe1-42f4-8fee-f69513af1ea8","Type":"ContainerStarted","Data":"b0389ff0e7cc20a6ea48b635b5dc5d3261c5e2d0485ace78d9bfdc08ab15d1da"} Oct 30 22:29:07.916995 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:07.916961 5949 generic.go:356] "Generic (PLEG): container finished" podID="76c39cd8-6fe1-42f4-8fee-f69513af1ea8" containerID="c00ae369b81d9d3125802a4016559b297d2042fccd115fa661ca5435118f2ce5" exitCode=0 Oct 30 22:29:07.917461 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:07.917032 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" event={"ID":"76c39cd8-6fe1-42f4-8fee-f69513af1ea8","Type":"ContainerDied","Data":"c00ae369b81d9d3125802a4016559b297d2042fccd115fa661ca5435118f2ce5"} Oct 30 22:29:08.922793 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:08.922759 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" event={"ID":"76c39cd8-6fe1-42f4-8fee-f69513af1ea8","Type":"ContainerStarted","Data":"d219d7976c7c382b653222f437f854e22780136aaa8b1e15d8372522fc938874"} Oct 30 22:29:08.923282 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:08.922831 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" event={"ID":"76c39cd8-6fe1-42f4-8fee-f69513af1ea8","Type":"ContainerStarted","Data":"dcf9925970ad0eaa0ad3850e85f04ee047e39de8bb00fece8c3e813aea7731a4"} Oct 30 22:29:08.923282 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:08.923059 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" Oct 30 22:29:08.977627 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:08.977579 5949 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" podStartSLOduration=3.089496572 podStartE2EDuration="6.977566132s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.88806956s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:29:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:29:08.977020516 +0000 UTC m=+1236.940829105" watchObservedRunningTime="2026-10-30 22:29:08.977566132 +0000 UTC m=+1236.941374679" Oct 30 22:29:09.926463 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:09.926431 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" Oct 30 22:29:09.927707 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:09.927678 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" podUID="76c39cd8-6fe1-42f4-8fee-f69513af1ea8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Oct 30 22:29:10.930307 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:10.930266 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" podUID="76c39cd8-6fe1-42f4-8fee-f69513af1ea8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Oct 30 22:29:15.934782 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:15.934753 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" Oct 30 22:29:15.935283 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:15.935256 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" podUID="76c39cd8-6fe1-42f4-8fee-f69513af1ea8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Oct 30 22:29:25.336244 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:29:25.336194 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:29:25.936203 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:25.936166 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" podUID="76c39cd8-6fe1-42f4-8fee-f69513af1ea8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Oct 30 22:29:35.935831 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:35.935791 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" podUID="76c39cd8-6fe1-42f4-8fee-f69513af1ea8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Oct 30 22:29:45.935505 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:45.935422 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" podUID="76c39cd8-6fe1-42f4-8fee-f69513af1ea8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Oct 30 22:29:55.335936 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:29:55.335893 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:29:55.935945 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:29:55.935908 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" podUID="76c39cd8-6fe1-42f4-8fee-f69513af1ea8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Oct 30 22:30:05.935402 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:05.935360 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" podUID="76c39cd8-6fe1-42f4-8fee-f69513af1ea8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Oct 30 22:30:15.936169 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:15.936128 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" podUID="76c39cd8-6fe1-42f4-8fee-f69513af1ea8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Oct 30 22:30:25.335964 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:30:25.335925 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:30:25.935387 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:25.935353 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" Oct 30 22:30:33.242433 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:33.242396 5949 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7"] Oct 30 22:30:33.242936 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:33.242634 5949 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" podUID="76c39cd8-6fe1-42f4-8fee-f69513af1ea8" containerName="kserve-container" containerID="cri-o://dcf9925970ad0eaa0ad3850e85f04ee047e39de8bb00fece8c3e813aea7731a4" gracePeriod=30 Oct 30 22:30:33.242936 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:33.242703 5949 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" podUID="76c39cd8-6fe1-42f4-8fee-f69513af1ea8" containerName="kube-rbac-proxy" containerID="cri-o://d219d7976c7c382b653222f437f854e22780136aaa8b1e15d8372522fc938874" gracePeriod=30 Oct 30 22:30:34.237269 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:34.237237 5949 generic.go:356] "Generic (PLEG): container finished" podID="76c39cd8-6fe1-42f4-8fee-f69513af1ea8" containerID="d219d7976c7c382b653222f437f854e22780136aaa8b1e15d8372522fc938874" exitCode=0 Oct 30 22:30:34.237441 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:34.237288 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" event={"ID":"76c39cd8-6fe1-42f4-8fee-f69513af1ea8","Type":"ContainerDied","Data":"d219d7976c7c382b653222f437f854e22780136aaa8b1e15d8372522fc938874"} Oct 30 22:30:35.930548 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:35.930504 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" podUID="76c39cd8-6fe1-42f4-8fee-f69513af1ea8" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.28:8643/healthz\": dial tcp 10.134.0.28:8643: connect: connection refused" Oct 30 22:30:35.935935 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:35.935909 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" podUID="76c39cd8-6fe1-42f4-8fee-f69513af1ea8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Oct 30 22:30:40.930903 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:40.930858 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" podUID="76c39cd8-6fe1-42f4-8fee-f69513af1ea8" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.28:8643/healthz\": dial tcp 10.134.0.28:8643: connect: connection refused" Oct 30 22:30:41.680537 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:41.680511 5949 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" Oct 30 22:30:41.777357 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:41.777283 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/76c39cd8-6fe1-42f4-8fee-f69513af1ea8-proxy-tls\" (UniqueName: \"kubernetes.io/secret/76c39cd8-6fe1-42f4-8fee-f69513af1ea8-proxy-tls\") pod \"76c39cd8-6fe1-42f4-8fee-f69513af1ea8\" (UID: \"76c39cd8-6fe1-42f4-8fee-f69513af1ea8\") " Oct 30 22:30:41.777357 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:41.777321 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/76c39cd8-6fe1-42f4-8fee-f69513af1ea8-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/76c39cd8-6fe1-42f4-8fee-f69513af1ea8-kube-rbac-proxy-sar-config\") pod \"76c39cd8-6fe1-42f4-8fee-f69513af1ea8\" (UID: \"76c39cd8-6fe1-42f4-8fee-f69513af1ea8\") " Oct 30 22:30:41.777567 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:41.777383 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76c39cd8-6fe1-42f4-8fee-f69513af1ea8-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/76c39cd8-6fe1-42f4-8fee-f69513af1ea8-kserve-provision-location\") pod \"76c39cd8-6fe1-42f4-8fee-f69513af1ea8\" (UID: \"76c39cd8-6fe1-42f4-8fee-f69513af1ea8\") " Oct 30 22:30:41.777567 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:41.777412 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/76c39cd8-6fe1-42f4-8fee-f69513af1ea8-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/76c39cd8-6fe1-42f4-8fee-f69513af1ea8-cabundle-cert\") pod \"76c39cd8-6fe1-42f4-8fee-f69513af1ea8\" (UID: \"76c39cd8-6fe1-42f4-8fee-f69513af1ea8\") " Oct 30 22:30:41.777567 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:41.777449 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/76c39cd8-6fe1-42f4-8fee-f69513af1ea8-kube-api-access-7766l\" (UniqueName: \"kubernetes.io/projected/76c39cd8-6fe1-42f4-8fee-f69513af1ea8-kube-api-access-7766l\") pod \"76c39cd8-6fe1-42f4-8fee-f69513af1ea8\" (UID: \"76c39cd8-6fe1-42f4-8fee-f69513af1ea8\") " Oct 30 22:30:41.777770 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:41.777743 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76c39cd8-6fe1-42f4-8fee-f69513af1ea8-kserve-provision-location" pod "76c39cd8-6fe1-42f4-8fee-f69513af1ea8" (UID: "76c39cd8-6fe1-42f4-8fee-f69513af1ea8"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 22:30:41.777842 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:41.777754 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76c39cd8-6fe1-42f4-8fee-f69513af1ea8-kube-rbac-proxy-sar-config" pod "76c39cd8-6fe1-42f4-8fee-f69513af1ea8" (UID: "76c39cd8-6fe1-42f4-8fee-f69513af1ea8"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:30:41.777946 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:41.777923 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76c39cd8-6fe1-42f4-8fee-f69513af1ea8-cabundle-cert" pod "76c39cd8-6fe1-42f4-8fee-f69513af1ea8" (UID: "76c39cd8-6fe1-42f4-8fee-f69513af1ea8"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:30:41.779454 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:41.779427 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76c39cd8-6fe1-42f4-8fee-f69513af1ea8-proxy-tls" pod "76c39cd8-6fe1-42f4-8fee-f69513af1ea8" (UID: "76c39cd8-6fe1-42f4-8fee-f69513af1ea8"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:30:41.779635 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:41.779615 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76c39cd8-6fe1-42f4-8fee-f69513af1ea8-kube-api-access-7766l" pod "76c39cd8-6fe1-42f4-8fee-f69513af1ea8" (UID: "76c39cd8-6fe1-42f4-8fee-f69513af1ea8"). InnerVolumeSpecName "kube-api-access-7766l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 22:30:41.878728 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:41.878701 5949 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/76c39cd8-6fe1-42f4-8fee-f69513af1ea8-kserve-provision-location\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:30:41.878728 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:41.878722 5949 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/76c39cd8-6fe1-42f4-8fee-f69513af1ea8-cabundle-cert\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:30:41.878870 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:41.878746 5949 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7766l\" (UniqueName: \"kubernetes.io/projected/76c39cd8-6fe1-42f4-8fee-f69513af1ea8-kube-api-access-7766l\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:30:41.878870 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:41.878755 5949 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/76c39cd8-6fe1-42f4-8fee-f69513af1ea8-proxy-tls\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:30:41.878870 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:41.878764 5949 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/76c39cd8-6fe1-42f4-8fee-f69513af1ea8-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:30:42.267687 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:42.267647 5949 generic.go:356] "Generic (PLEG): container finished" podID="76c39cd8-6fe1-42f4-8fee-f69513af1ea8" containerID="dcf9925970ad0eaa0ad3850e85f04ee047e39de8bb00fece8c3e813aea7731a4" exitCode=0 Oct 30 22:30:42.268073 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:42.267714 5949 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" Oct 30 22:30:42.268073 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:42.267720 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" event={"ID":"76c39cd8-6fe1-42f4-8fee-f69513af1ea8","Type":"ContainerDied","Data":"dcf9925970ad0eaa0ad3850e85f04ee047e39de8bb00fece8c3e813aea7731a4"} Oct 30 22:30:42.268073 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:42.267778 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7" event={"ID":"76c39cd8-6fe1-42f4-8fee-f69513af1ea8","Type":"ContainerDied","Data":"b0389ff0e7cc20a6ea48b635b5dc5d3261c5e2d0485ace78d9bfdc08ab15d1da"} Oct 30 22:30:42.268073 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:42.267805 5949 scope.go:122] "RemoveContainer" containerID="d219d7976c7c382b653222f437f854e22780136aaa8b1e15d8372522fc938874" Oct 30 22:30:42.275549 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:42.275532 5949 scope.go:122] "RemoveContainer" containerID="dcf9925970ad0eaa0ad3850e85f04ee047e39de8bb00fece8c3e813aea7731a4" Oct 30 22:30:42.282783 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:42.282764 5949 scope.go:122] "RemoveContainer" containerID="c00ae369b81d9d3125802a4016559b297d2042fccd115fa661ca5435118f2ce5" Oct 30 22:30:42.289715 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:42.289698 5949 scope.go:122] "RemoveContainer" containerID="d219d7976c7c382b653222f437f854e22780136aaa8b1e15d8372522fc938874" Oct 30 22:30:42.289971 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:30:42.289949 5949 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d219d7976c7c382b653222f437f854e22780136aaa8b1e15d8372522fc938874\": container with ID starting with d219d7976c7c382b653222f437f854e22780136aaa8b1e15d8372522fc938874 not found: ID does not exist" containerID="d219d7976c7c382b653222f437f854e22780136aaa8b1e15d8372522fc938874" Oct 30 22:30:42.290056 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:42.289975 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d219d7976c7c382b653222f437f854e22780136aaa8b1e15d8372522fc938874"} err="failed to get container status \"d219d7976c7c382b653222f437f854e22780136aaa8b1e15d8372522fc938874\": rpc error: code = NotFound desc = could not find container \"d219d7976c7c382b653222f437f854e22780136aaa8b1e15d8372522fc938874\": container with ID starting with d219d7976c7c382b653222f437f854e22780136aaa8b1e15d8372522fc938874 not found: ID does not exist" Oct 30 22:30:42.290056 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:42.289995 5949 scope.go:122] "RemoveContainer" containerID="dcf9925970ad0eaa0ad3850e85f04ee047e39de8bb00fece8c3e813aea7731a4" Oct 30 22:30:42.290242 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:30:42.290207 5949 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcf9925970ad0eaa0ad3850e85f04ee047e39de8bb00fece8c3e813aea7731a4\": container with ID starting with dcf9925970ad0eaa0ad3850e85f04ee047e39de8bb00fece8c3e813aea7731a4 not found: ID does not exist" containerID="dcf9925970ad0eaa0ad3850e85f04ee047e39de8bb00fece8c3e813aea7731a4" Oct 30 22:30:42.290334 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:42.290246 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcf9925970ad0eaa0ad3850e85f04ee047e39de8bb00fece8c3e813aea7731a4"} err="failed to get container status \"dcf9925970ad0eaa0ad3850e85f04ee047e39de8bb00fece8c3e813aea7731a4\": rpc error: code = NotFound desc = could not find container \"dcf9925970ad0eaa0ad3850e85f04ee047e39de8bb00fece8c3e813aea7731a4\": container with ID starting with dcf9925970ad0eaa0ad3850e85f04ee047e39de8bb00fece8c3e813aea7731a4 not found: ID does not exist" Oct 30 22:30:42.290334 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:42.290262 5949 scope.go:122] "RemoveContainer" containerID="c00ae369b81d9d3125802a4016559b297d2042fccd115fa661ca5435118f2ce5" Oct 30 22:30:42.290513 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:30:42.290495 5949 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c00ae369b81d9d3125802a4016559b297d2042fccd115fa661ca5435118f2ce5\": container with ID starting with c00ae369b81d9d3125802a4016559b297d2042fccd115fa661ca5435118f2ce5 not found: ID does not exist" containerID="c00ae369b81d9d3125802a4016559b297d2042fccd115fa661ca5435118f2ce5" Oct 30 22:30:42.290550 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:42.290519 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c00ae369b81d9d3125802a4016559b297d2042fccd115fa661ca5435118f2ce5"} err="failed to get container status \"c00ae369b81d9d3125802a4016559b297d2042fccd115fa661ca5435118f2ce5\": rpc error: code = NotFound desc = could not find container \"c00ae369b81d9d3125802a4016559b297d2042fccd115fa661ca5435118f2ce5\": container with ID starting with c00ae369b81d9d3125802a4016559b297d2042fccd115fa661ca5435118f2ce5 not found: ID does not exist" Oct 30 22:30:42.333536 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:42.333511 5949 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7"] Oct 30 22:30:42.345011 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:42.344988 5949 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-306b77ff42/raw-sklearn-33c9b-predictor-5c4b5d5b5b-cftk7"] Oct 30 22:30:44.159762 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:44.159728 5949 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="76c39cd8-6fe1-42f4-8fee-f69513af1ea8" path="/var/lib/kubelet/pods/76c39cd8-6fe1-42f4-8fee-f69513af1ea8/volumes" Oct 30 22:30:44.927747 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:44.927708 5949 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j"] Oct 30 22:30:44.928088 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:44.928074 5949 cpu_manager.go:414] "containerMap: removing container" podUID="76c39cd8-6fe1-42f4-8fee-f69513af1ea8" containerName="kube-rbac-proxy" Oct 30 22:30:44.928156 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:44.928090 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="76c39cd8-6fe1-42f4-8fee-f69513af1ea8" containerName="kube-rbac-proxy" Oct 30 22:30:44.928156 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:44.928112 5949 cpu_manager.go:414] "containerMap: removing container" podUID="76c39cd8-6fe1-42f4-8fee-f69513af1ea8" containerName="kserve-container" Oct 30 22:30:44.928156 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:44.928117 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="76c39cd8-6fe1-42f4-8fee-f69513af1ea8" containerName="kserve-container" Oct 30 22:30:44.928156 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:44.928132 5949 cpu_manager.go:414] "containerMap: removing container" podUID="76c39cd8-6fe1-42f4-8fee-f69513af1ea8" containerName="storage-initializer" Oct 30 22:30:44.928156 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:44.928137 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="76c39cd8-6fe1-42f4-8fee-f69513af1ea8" containerName="storage-initializer" Oct 30 22:30:44.928343 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:44.928194 5949 memory_manager.go:365] "RemoveStaleState removing state" podUID="76c39cd8-6fe1-42f4-8fee-f69513af1ea8" containerName="kube-rbac-proxy" Oct 30 22:30:44.928343 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:44.928204 5949 memory_manager.go:365] "RemoveStaleState removing state" podUID="76c39cd8-6fe1-42f4-8fee-f69513af1ea8" containerName="kserve-container" Oct 30 22:30:44.932883 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:44.932862 5949 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" Oct 30 22:30:44.948903 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:44.948884 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-306b77ff42\"/\"seaweedfs-s3-creds\"" Oct 30 22:30:44.959806 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:44.959785 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-306b77ff42\"/\"raw-sklearn-runtime-2cd7c-predictor-serving-cert\"" Oct 30 22:30:44.959942 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:44.959853 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-306b77ff42\"/\"default-dockercfg-7rhfl\"" Oct 30 22:30:44.959942 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:44.959872 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-306b77ff42\"/\"openshift-service-ca.crt\"" Oct 30 22:30:44.959942 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:44.959927 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-306b77ff42\"/\"kube-root-ca.crt\"" Oct 30 22:30:44.960309 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:44.960286 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-306b77ff42\"/\"raw-sklearn-runtime-2cd7c-kube-rbac-proxy-sar-config\"" Oct 30 22:30:44.960473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:44.960353 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-306b77ff42\"/\"odh-kserve-custom-ca-bundle\"" Oct 30 22:30:44.961545 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:44.961526 5949 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j"] Oct 30 22:30:45.003578 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:45.003548 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzhds\" (UniqueName: \"kubernetes.io/projected/5f324d8a-587b-4169-90a0-1c182c0f3dfc-kube-api-access-xzhds\") pod \"raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j\" (UID: \"5f324d8a-587b-4169-90a0-1c182c0f3dfc\") " pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" Oct 30 22:30:45.003719 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:45.003595 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5f324d8a-587b-4169-90a0-1c182c0f3dfc-kserve-provision-location\") pod \"raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j\" (UID: \"5f324d8a-587b-4169-90a0-1c182c0f3dfc\") " pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" Oct 30 22:30:45.003719 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:45.003701 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5f324d8a-587b-4169-90a0-1c182c0f3dfc-proxy-tls\") pod \"raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j\" (UID: \"5f324d8a-587b-4169-90a0-1c182c0f3dfc\") " pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" Oct 30 22:30:45.003791 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:45.003748 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/5f324d8a-587b-4169-90a0-1c182c0f3dfc-cabundle-cert\") pod \"raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j\" (UID: \"5f324d8a-587b-4169-90a0-1c182c0f3dfc\") " pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" Oct 30 22:30:45.003791 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:45.003774 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5f324d8a-587b-4169-90a0-1c182c0f3dfc-kube-rbac-proxy-sar-config\") pod \"raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j\" (UID: \"5f324d8a-587b-4169-90a0-1c182c0f3dfc\") " pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" Oct 30 22:30:45.104779 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:45.104741 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/5f324d8a-587b-4169-90a0-1c182c0f3dfc-cabundle-cert\") pod \"raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j\" (UID: \"5f324d8a-587b-4169-90a0-1c182c0f3dfc\") " pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" Oct 30 22:30:45.104974 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:45.104789 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5f324d8a-587b-4169-90a0-1c182c0f3dfc-kube-rbac-proxy-sar-config\") pod \"raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j\" (UID: \"5f324d8a-587b-4169-90a0-1c182c0f3dfc\") " pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" Oct 30 22:30:45.104974 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:45.104819 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzhds\" (UniqueName: \"kubernetes.io/projected/5f324d8a-587b-4169-90a0-1c182c0f3dfc-kube-api-access-xzhds\") pod \"raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j\" (UID: \"5f324d8a-587b-4169-90a0-1c182c0f3dfc\") " pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" Oct 30 22:30:45.104974 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:45.104857 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5f324d8a-587b-4169-90a0-1c182c0f3dfc-kserve-provision-location\") pod \"raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j\" (UID: \"5f324d8a-587b-4169-90a0-1c182c0f3dfc\") " pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" Oct 30 22:30:45.104974 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:45.104924 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5f324d8a-587b-4169-90a0-1c182c0f3dfc-proxy-tls\") pod \"raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j\" (UID: \"5f324d8a-587b-4169-90a0-1c182c0f3dfc\") " pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" Oct 30 22:30:45.105308 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:45.105283 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5f324d8a-587b-4169-90a0-1c182c0f3dfc-kserve-provision-location\") pod \"raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j\" (UID: \"5f324d8a-587b-4169-90a0-1c182c0f3dfc\") " pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" Oct 30 22:30:45.105572 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:45.105550 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/5f324d8a-587b-4169-90a0-1c182c0f3dfc-cabundle-cert\") pod \"raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j\" (UID: \"5f324d8a-587b-4169-90a0-1c182c0f3dfc\") " pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" Oct 30 22:30:45.105639 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:45.105575 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5f324d8a-587b-4169-90a0-1c182c0f3dfc-kube-rbac-proxy-sar-config\") pod \"raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j\" (UID: \"5f324d8a-587b-4169-90a0-1c182c0f3dfc\") " pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" Oct 30 22:30:45.107313 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:45.107293 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5f324d8a-587b-4169-90a0-1c182c0f3dfc-proxy-tls\") pod \"raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j\" (UID: \"5f324d8a-587b-4169-90a0-1c182c0f3dfc\") " pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" Oct 30 22:30:45.133246 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:45.133211 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzhds\" (UniqueName: \"kubernetes.io/projected/5f324d8a-587b-4169-90a0-1c182c0f3dfc-kube-api-access-xzhds\") pod \"raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j\" (UID: \"5f324d8a-587b-4169-90a0-1c182c0f3dfc\") " pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" Oct 30 22:30:45.249586 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:45.249505 5949 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" Oct 30 22:30:45.556164 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:45.556083 5949 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j"] Oct 30 22:30:45.561555 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:30:45.561529 5949 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f324d8a_587b_4169_90a0_1c182c0f3dfc.slice/crio-119534711f6155082602523f1e1b320ccef23b3397eec073705c5d700460c3dc WatchSource:0}: Error finding container 119534711f6155082602523f1e1b320ccef23b3397eec073705c5d700460c3dc: Status 404 returned error can't find the container with id 119534711f6155082602523f1e1b320ccef23b3397eec073705c5d700460c3dc Oct 30 22:30:46.285883 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:46.285841 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" event={"ID":"5f324d8a-587b-4169-90a0-1c182c0f3dfc","Type":"ContainerStarted","Data":"501a7aae43e13671b54ed830aa9553b9f774e9844f1fe6b43cefb390ee506754"} Oct 30 22:30:46.286309 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:46.285912 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" event={"ID":"5f324d8a-587b-4169-90a0-1c182c0f3dfc","Type":"ContainerStarted","Data":"119534711f6155082602523f1e1b320ccef23b3397eec073705c5d700460c3dc"} Oct 30 22:30:50.301183 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:50.301145 5949 generic.go:356] "Generic (PLEG): container finished" podID="5f324d8a-587b-4169-90a0-1c182c0f3dfc" containerID="501a7aae43e13671b54ed830aa9553b9f774e9844f1fe6b43cefb390ee506754" exitCode=0 Oct 30 22:30:50.301597 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:50.301233 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" event={"ID":"5f324d8a-587b-4169-90a0-1c182c0f3dfc","Type":"ContainerDied","Data":"501a7aae43e13671b54ed830aa9553b9f774e9844f1fe6b43cefb390ee506754"} Oct 30 22:30:51.308099 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:51.308065 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" event={"ID":"5f324d8a-587b-4169-90a0-1c182c0f3dfc","Type":"ContainerStarted","Data":"e1c22e9dbb0b2f6793981696523db153755c896a38e7fb29c071b18aca44ef16"} Oct 30 22:30:51.308518 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:51.308122 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" event={"ID":"5f324d8a-587b-4169-90a0-1c182c0f3dfc","Type":"ContainerStarted","Data":"96166d4237eecc23ba5aa2a86d823dd84a106b3cdc500460e7909e828397fbb6"} Oct 30 22:30:51.308518 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:51.308335 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" Oct 30 22:30:51.364655 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:51.364607 5949 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" podStartSLOduration=3.321409385 podStartE2EDuration="7.36459558s" totalImagesPullingTime="0s" totalInitContainerRuntime="4.043186195s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:30:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:30:51.364078358 +0000 UTC m=+1339.327886924" watchObservedRunningTime="2026-10-30 22:30:51.36459558 +0000 UTC m=+1339.328404149" Oct 30 22:30:52.312297 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:52.312263 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" Oct 30 22:30:52.313530 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:52.313502 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" podUID="5f324d8a-587b-4169-90a0-1c182c0f3dfc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused" Oct 30 22:30:53.316570 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:53.316532 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" podUID="5f324d8a-587b-4169-90a0-1c182c0f3dfc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused" Oct 30 22:30:55.336381 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:30:55.336342 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:30:58.321410 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:58.321380 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" Oct 30 22:30:58.322043 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:30:58.322014 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" podUID="5f324d8a-587b-4169-90a0-1c182c0f3dfc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused" Oct 30 22:31:08.322894 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:31:08.322854 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" podUID="5f324d8a-587b-4169-90a0-1c182c0f3dfc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused" Oct 30 22:31:18.322172 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:31:18.322095 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" podUID="5f324d8a-587b-4169-90a0-1c182c0f3dfc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused" Oct 30 22:31:25.335685 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:31:25.335651 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:31:28.322907 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:31:28.322870 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" podUID="5f324d8a-587b-4169-90a0-1c182c0f3dfc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused" Oct 30 22:31:38.322345 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:31:38.322306 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" podUID="5f324d8a-587b-4169-90a0-1c182c0f3dfc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused" Oct 30 22:31:48.322928 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:31:48.322891 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" podUID="5f324d8a-587b-4169-90a0-1c182c0f3dfc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused" Oct 30 22:31:55.335860 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:31:55.335824 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:31:58.322131 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:31:58.322090 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" podUID="5f324d8a-587b-4169-90a0-1c182c0f3dfc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused" Oct 30 22:32:08.323379 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:08.323350 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" Oct 30 22:32:15.716537 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:15.716507 5949 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j"] Oct 30 22:32:15.716982 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:15.716767 5949 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" podUID="5f324d8a-587b-4169-90a0-1c182c0f3dfc" containerName="kserve-container" containerID="cri-o://96166d4237eecc23ba5aa2a86d823dd84a106b3cdc500460e7909e828397fbb6" gracePeriod=30 Oct 30 22:32:15.716982 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:15.716805 5949 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" podUID="5f324d8a-587b-4169-90a0-1c182c0f3dfc" containerName="kube-rbac-proxy" containerID="cri-o://e1c22e9dbb0b2f6793981696523db153755c896a38e7fb29c071b18aca44ef16" gracePeriod=30 Oct 30 22:32:16.628295 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:16.628261 5949 generic.go:356] "Generic (PLEG): container finished" podID="5f324d8a-587b-4169-90a0-1c182c0f3dfc" containerID="e1c22e9dbb0b2f6793981696523db153755c896a38e7fb29c071b18aca44ef16" exitCode=0 Oct 30 22:32:16.628463 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:16.628330 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" event={"ID":"5f324d8a-587b-4169-90a0-1c182c0f3dfc","Type":"ContainerDied","Data":"e1c22e9dbb0b2f6793981696523db153755c896a38e7fb29c071b18aca44ef16"} Oct 30 22:32:18.316859 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:18.316824 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" podUID="5f324d8a-587b-4169-90a0-1c182c0f3dfc" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.29:8643/healthz\": dial tcp 10.134.0.29:8643: connect: connection refused" Oct 30 22:32:18.322524 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:18.322499 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" podUID="5f324d8a-587b-4169-90a0-1c182c0f3dfc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused" Oct 30 22:32:20.560441 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:20.560411 5949 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" Oct 30 22:32:20.599614 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:20.599584 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5f324d8a-587b-4169-90a0-1c182c0f3dfc-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/5f324d8a-587b-4169-90a0-1c182c0f3dfc-cabundle-cert\") pod \"5f324d8a-587b-4169-90a0-1c182c0f3dfc\" (UID: \"5f324d8a-587b-4169-90a0-1c182c0f3dfc\") " Oct 30 22:32:20.599743 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:20.599626 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5f324d8a-587b-4169-90a0-1c182c0f3dfc-proxy-tls\" (UniqueName: \"kubernetes.io/secret/5f324d8a-587b-4169-90a0-1c182c0f3dfc-proxy-tls\") pod \"5f324d8a-587b-4169-90a0-1c182c0f3dfc\" (UID: \"5f324d8a-587b-4169-90a0-1c182c0f3dfc\") " Oct 30 22:32:20.599743 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:20.599675 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5f324d8a-587b-4169-90a0-1c182c0f3dfc-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5f324d8a-587b-4169-90a0-1c182c0f3dfc-kserve-provision-location\") pod \"5f324d8a-587b-4169-90a0-1c182c0f3dfc\" (UID: \"5f324d8a-587b-4169-90a0-1c182c0f3dfc\") " Oct 30 22:32:20.599743 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:20.599713 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5f324d8a-587b-4169-90a0-1c182c0f3dfc-kube-api-access-xzhds\" (UniqueName: \"kubernetes.io/projected/5f324d8a-587b-4169-90a0-1c182c0f3dfc-kube-api-access-xzhds\") pod \"5f324d8a-587b-4169-90a0-1c182c0f3dfc\" (UID: \"5f324d8a-587b-4169-90a0-1c182c0f3dfc\") " Oct 30 22:32:20.599863 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:20.599757 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5f324d8a-587b-4169-90a0-1c182c0f3dfc-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5f324d8a-587b-4169-90a0-1c182c0f3dfc-kube-rbac-proxy-sar-config\") pod \"5f324d8a-587b-4169-90a0-1c182c0f3dfc\" (UID: \"5f324d8a-587b-4169-90a0-1c182c0f3dfc\") " Oct 30 22:32:20.600022 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:20.599994 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f324d8a-587b-4169-90a0-1c182c0f3dfc-cabundle-cert" pod "5f324d8a-587b-4169-90a0-1c182c0f3dfc" (UID: "5f324d8a-587b-4169-90a0-1c182c0f3dfc"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:32:20.600085 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:20.600006 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f324d8a-587b-4169-90a0-1c182c0f3dfc-kserve-provision-location" pod "5f324d8a-587b-4169-90a0-1c182c0f3dfc" (UID: "5f324d8a-587b-4169-90a0-1c182c0f3dfc"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 22:32:20.600199 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:20.600179 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f324d8a-587b-4169-90a0-1c182c0f3dfc-kube-rbac-proxy-sar-config" pod "5f324d8a-587b-4169-90a0-1c182c0f3dfc" (UID: "5f324d8a-587b-4169-90a0-1c182c0f3dfc"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:32:20.601770 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:20.601750 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f324d8a-587b-4169-90a0-1c182c0f3dfc-proxy-tls" pod "5f324d8a-587b-4169-90a0-1c182c0f3dfc" (UID: "5f324d8a-587b-4169-90a0-1c182c0f3dfc"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:32:20.601885 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:20.601865 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f324d8a-587b-4169-90a0-1c182c0f3dfc-kube-api-access-xzhds" pod "5f324d8a-587b-4169-90a0-1c182c0f3dfc" (UID: "5f324d8a-587b-4169-90a0-1c182c0f3dfc"). InnerVolumeSpecName "kube-api-access-xzhds". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 22:32:20.644134 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:20.644054 5949 generic.go:356] "Generic (PLEG): container finished" podID="5f324d8a-587b-4169-90a0-1c182c0f3dfc" containerID="96166d4237eecc23ba5aa2a86d823dd84a106b3cdc500460e7909e828397fbb6" exitCode=0 Oct 30 22:32:20.644134 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:20.644119 5949 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" Oct 30 22:32:20.644363 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:20.644133 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" event={"ID":"5f324d8a-587b-4169-90a0-1c182c0f3dfc","Type":"ContainerDied","Data":"96166d4237eecc23ba5aa2a86d823dd84a106b3cdc500460e7909e828397fbb6"} Oct 30 22:32:20.644363 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:20.644196 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j" event={"ID":"5f324d8a-587b-4169-90a0-1c182c0f3dfc","Type":"ContainerDied","Data":"119534711f6155082602523f1e1b320ccef23b3397eec073705c5d700460c3dc"} Oct 30 22:32:20.644363 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:20.644241 5949 scope.go:122] "RemoveContainer" containerID="e1c22e9dbb0b2f6793981696523db153755c896a38e7fb29c071b18aca44ef16" Oct 30 22:32:20.652874 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:20.652858 5949 scope.go:122] "RemoveContainer" containerID="96166d4237eecc23ba5aa2a86d823dd84a106b3cdc500460e7909e828397fbb6" Oct 30 22:32:20.660090 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:20.660075 5949 scope.go:122] "RemoveContainer" containerID="501a7aae43e13671b54ed830aa9553b9f774e9844f1fe6b43cefb390ee506754" Oct 30 22:32:20.667185 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:20.667170 5949 scope.go:122] "RemoveContainer" containerID="e1c22e9dbb0b2f6793981696523db153755c896a38e7fb29c071b18aca44ef16" Oct 30 22:32:20.667457 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:32:20.667438 5949 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1c22e9dbb0b2f6793981696523db153755c896a38e7fb29c071b18aca44ef16\": container with ID starting with e1c22e9dbb0b2f6793981696523db153755c896a38e7fb29c071b18aca44ef16 not found: ID does not exist" containerID="e1c22e9dbb0b2f6793981696523db153755c896a38e7fb29c071b18aca44ef16" Oct 30 22:32:20.667525 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:20.667464 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1c22e9dbb0b2f6793981696523db153755c896a38e7fb29c071b18aca44ef16"} err="failed to get container status \"e1c22e9dbb0b2f6793981696523db153755c896a38e7fb29c071b18aca44ef16\": rpc error: code = NotFound desc = could not find container \"e1c22e9dbb0b2f6793981696523db153755c896a38e7fb29c071b18aca44ef16\": container with ID starting with e1c22e9dbb0b2f6793981696523db153755c896a38e7fb29c071b18aca44ef16 not found: ID does not exist" Oct 30 22:32:20.667525 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:20.667480 5949 scope.go:122] "RemoveContainer" containerID="96166d4237eecc23ba5aa2a86d823dd84a106b3cdc500460e7909e828397fbb6" Oct 30 22:32:20.667696 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:32:20.667682 5949 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96166d4237eecc23ba5aa2a86d823dd84a106b3cdc500460e7909e828397fbb6\": container with ID starting with 96166d4237eecc23ba5aa2a86d823dd84a106b3cdc500460e7909e828397fbb6 not found: ID does not exist" containerID="96166d4237eecc23ba5aa2a86d823dd84a106b3cdc500460e7909e828397fbb6" Oct 30 22:32:20.667737 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:20.667700 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96166d4237eecc23ba5aa2a86d823dd84a106b3cdc500460e7909e828397fbb6"} err="failed to get container status \"96166d4237eecc23ba5aa2a86d823dd84a106b3cdc500460e7909e828397fbb6\": rpc error: code = NotFound desc = could not find container \"96166d4237eecc23ba5aa2a86d823dd84a106b3cdc500460e7909e828397fbb6\": container with ID starting with 96166d4237eecc23ba5aa2a86d823dd84a106b3cdc500460e7909e828397fbb6 not found: ID does not exist" Oct 30 22:32:20.667737 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:20.667712 5949 scope.go:122] "RemoveContainer" containerID="501a7aae43e13671b54ed830aa9553b9f774e9844f1fe6b43cefb390ee506754" Oct 30 22:32:20.667910 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:32:20.667895 5949 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"501a7aae43e13671b54ed830aa9553b9f774e9844f1fe6b43cefb390ee506754\": container with ID starting with 501a7aae43e13671b54ed830aa9553b9f774e9844f1fe6b43cefb390ee506754 not found: ID does not exist" containerID="501a7aae43e13671b54ed830aa9553b9f774e9844f1fe6b43cefb390ee506754" Oct 30 22:32:20.667949 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:20.667913 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"501a7aae43e13671b54ed830aa9553b9f774e9844f1fe6b43cefb390ee506754"} err="failed to get container status \"501a7aae43e13671b54ed830aa9553b9f774e9844f1fe6b43cefb390ee506754\": rpc error: code = NotFound desc = could not find container \"501a7aae43e13671b54ed830aa9553b9f774e9844f1fe6b43cefb390ee506754\": container with ID starting with 501a7aae43e13671b54ed830aa9553b9f774e9844f1fe6b43cefb390ee506754 not found: ID does not exist" Oct 30 22:32:20.695087 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:20.695065 5949 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j"] Oct 30 22:32:20.699393 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:20.699374 5949 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-306b77ff42/raw-sklearn-runtime-2cd7c-predictor-88cf84fcb-btq4j"] Oct 30 22:32:20.701173 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:20.701149 5949 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5f324d8a-587b-4169-90a0-1c182c0f3dfc-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:32:20.701173 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:20.701177 5949 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/5f324d8a-587b-4169-90a0-1c182c0f3dfc-cabundle-cert\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:32:20.701328 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:20.701192 5949 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5f324d8a-587b-4169-90a0-1c182c0f3dfc-proxy-tls\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:32:20.701328 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:20.701202 5949 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5f324d8a-587b-4169-90a0-1c182c0f3dfc-kserve-provision-location\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:32:20.701328 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:20.701240 5949 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xzhds\" (UniqueName: \"kubernetes.io/projected/5f324d8a-587b-4169-90a0-1c182c0f3dfc-kube-api-access-xzhds\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:32:22.159725 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:22.159694 5949 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5f324d8a-587b-4169-90a0-1c182c0f3dfc" path="/var/lib/kubelet/pods/5f324d8a-587b-4169-90a0-1c182c0f3dfc/volumes" Oct 30 22:32:23.353342 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.353303 5949 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7"] Oct 30 22:32:23.353738 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.353681 5949 cpu_manager.go:414] "containerMap: removing container" podUID="5f324d8a-587b-4169-90a0-1c182c0f3dfc" containerName="kube-rbac-proxy" Oct 30 22:32:23.353738 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.353695 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5f324d8a-587b-4169-90a0-1c182c0f3dfc" containerName="kube-rbac-proxy" Oct 30 22:32:23.353738 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.353707 5949 cpu_manager.go:414] "containerMap: removing container" podUID="5f324d8a-587b-4169-90a0-1c182c0f3dfc" containerName="kserve-container" Oct 30 22:32:23.353738 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.353712 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5f324d8a-587b-4169-90a0-1c182c0f3dfc" containerName="kserve-container" Oct 30 22:32:23.353738 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.353720 5949 cpu_manager.go:414] "containerMap: removing container" podUID="5f324d8a-587b-4169-90a0-1c182c0f3dfc" containerName="storage-initializer" Oct 30 22:32:23.353738 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.353726 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5f324d8a-587b-4169-90a0-1c182c0f3dfc" containerName="storage-initializer" Oct 30 22:32:23.353928 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.353772 5949 memory_manager.go:365] "RemoveStaleState removing state" podUID="5f324d8a-587b-4169-90a0-1c182c0f3dfc" containerName="kube-rbac-proxy" Oct 30 22:32:23.353928 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.353781 5949 memory_manager.go:365] "RemoveStaleState removing state" podUID="5f324d8a-587b-4169-90a0-1c182c0f3dfc" containerName="kserve-container" Oct 30 22:32:23.357066 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.357045 5949 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" Oct 30 22:32:23.386829 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.386806 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-306b77ff42\"/\"odh-kserve-custom-ca-bundle\"" Oct 30 22:32:23.386973 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.386858 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-306b77ff42\"/\"isvc-canary-create-kube-rbac-proxy-sar-config\"" Oct 30 22:32:23.386973 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.386868 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-306b77ff42\"/\"seaweedfs-s3-creds\"" Oct 30 22:32:23.386973 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.386868 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-306b77ff42\"/\"isvc-canary-create-v2-predictor-serving-cert\"" Oct 30 22:32:23.387282 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.387265 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-306b77ff42\"/\"openshift-service-ca.crt\"" Oct 30 22:32:23.387282 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.387274 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-306b77ff42\"/\"kube-root-ca.crt\"" Oct 30 22:32:23.387755 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.387741 5949 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7"] Oct 30 22:32:23.391771 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.391755 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-306b77ff42\"/\"default-dockercfg-7rhfl\"" Oct 30 22:32:23.423835 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.423811 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/18f63ac1-385a-4873-a4e9-49a2ecf97b38-kserve-provision-location\") pod \"isvc-canary-create-v2-predictor-86bcf9785b-mwzq7\" (UID: \"18f63ac1-385a-4873-a4e9-49a2ecf97b38\") " pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" Oct 30 22:32:23.423929 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.423859 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/18f63ac1-385a-4873-a4e9-49a2ecf97b38-cabundle-cert\") pod \"isvc-canary-create-v2-predictor-86bcf9785b-mwzq7\" (UID: \"18f63ac1-385a-4873-a4e9-49a2ecf97b38\") " pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" Oct 30 22:32:23.423969 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.423922 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/18f63ac1-385a-4873-a4e9-49a2ecf97b38-proxy-tls\") pod \"isvc-canary-create-v2-predictor-86bcf9785b-mwzq7\" (UID: \"18f63ac1-385a-4873-a4e9-49a2ecf97b38\") " pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" Oct 30 22:32:23.423969 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.423950 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/18f63ac1-385a-4873-a4e9-49a2ecf97b38-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-v2-predictor-86bcf9785b-mwzq7\" (UID: \"18f63ac1-385a-4873-a4e9-49a2ecf97b38\") " pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" Oct 30 22:32:23.424033 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.423986 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctdj4\" (UniqueName: \"kubernetes.io/projected/18f63ac1-385a-4873-a4e9-49a2ecf97b38-kube-api-access-ctdj4\") pod \"isvc-canary-create-v2-predictor-86bcf9785b-mwzq7\" (UID: \"18f63ac1-385a-4873-a4e9-49a2ecf97b38\") " pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" Oct 30 22:32:23.523799 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.523765 5949 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6"] Oct 30 22:32:23.525400 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.525368 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/18f63ac1-385a-4873-a4e9-49a2ecf97b38-kserve-provision-location\") pod \"isvc-canary-create-v2-predictor-86bcf9785b-mwzq7\" (UID: \"18f63ac1-385a-4873-a4e9-49a2ecf97b38\") " pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" Oct 30 22:32:23.525575 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.525431 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/18f63ac1-385a-4873-a4e9-49a2ecf97b38-cabundle-cert\") pod \"isvc-canary-create-v2-predictor-86bcf9785b-mwzq7\" (UID: \"18f63ac1-385a-4873-a4e9-49a2ecf97b38\") " pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" Oct 30 22:32:23.525575 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.525467 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/18f63ac1-385a-4873-a4e9-49a2ecf97b38-proxy-tls\") pod \"isvc-canary-create-v2-predictor-86bcf9785b-mwzq7\" (UID: \"18f63ac1-385a-4873-a4e9-49a2ecf97b38\") " pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" Oct 30 22:32:23.525575 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.525498 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/18f63ac1-385a-4873-a4e9-49a2ecf97b38-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-v2-predictor-86bcf9785b-mwzq7\" (UID: \"18f63ac1-385a-4873-a4e9-49a2ecf97b38\") " pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" Oct 30 22:32:23.525575 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.525527 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctdj4\" (UniqueName: \"kubernetes.io/projected/18f63ac1-385a-4873-a4e9-49a2ecf97b38-kube-api-access-ctdj4\") pod \"isvc-canary-create-v2-predictor-86bcf9785b-mwzq7\" (UID: \"18f63ac1-385a-4873-a4e9-49a2ecf97b38\") " pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" Oct 30 22:32:23.526253 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.526207 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/18f63ac1-385a-4873-a4e9-49a2ecf97b38-kserve-provision-location\") pod \"isvc-canary-create-v2-predictor-86bcf9785b-mwzq7\" (UID: \"18f63ac1-385a-4873-a4e9-49a2ecf97b38\") " pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" Oct 30 22:32:23.526384 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.526363 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/18f63ac1-385a-4873-a4e9-49a2ecf97b38-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-v2-predictor-86bcf9785b-mwzq7\" (UID: \"18f63ac1-385a-4873-a4e9-49a2ecf97b38\") " pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" Oct 30 22:32:23.526443 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.526409 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/18f63ac1-385a-4873-a4e9-49a2ecf97b38-cabundle-cert\") pod \"isvc-canary-create-v2-predictor-86bcf9785b-mwzq7\" (UID: \"18f63ac1-385a-4873-a4e9-49a2ecf97b38\") " pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" Oct 30 22:32:23.527720 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.527704 5949 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" Oct 30 22:32:23.528268 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.528245 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/18f63ac1-385a-4873-a4e9-49a2ecf97b38-proxy-tls\") pod \"isvc-canary-create-v2-predictor-86bcf9785b-mwzq7\" (UID: \"18f63ac1-385a-4873-a4e9-49a2ecf97b38\") " pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" Oct 30 22:32:23.551674 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.551653 5949 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6"] Oct 30 22:32:23.557821 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.557802 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctdj4\" (UniqueName: \"kubernetes.io/projected/18f63ac1-385a-4873-a4e9-49a2ecf97b38-kube-api-access-ctdj4\") pod \"isvc-canary-create-v2-predictor-86bcf9785b-mwzq7\" (UID: \"18f63ac1-385a-4873-a4e9-49a2ecf97b38\") " pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" Oct 30 22:32:23.576815 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.576790 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-306b77ff42\"/\"isvc-canary-create-predictor-serving-cert\"" Oct 30 22:32:23.626593 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.626542 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njcgt\" (UniqueName: \"kubernetes.io/projected/2cc674bf-c5d9-43c0-99b2-612e8b713ff8-kube-api-access-njcgt\") pod \"isvc-canary-create-predictor-54d47cbf97-x7vr6\" (UID: \"2cc674bf-c5d9-43c0-99b2-612e8b713ff8\") " pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" Oct 30 22:32:23.626671 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.626648 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2cc674bf-c5d9-43c0-99b2-612e8b713ff8-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-predictor-54d47cbf97-x7vr6\" (UID: \"2cc674bf-c5d9-43c0-99b2-612e8b713ff8\") " pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" Oct 30 22:32:23.626714 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.626677 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2cc674bf-c5d9-43c0-99b2-612e8b713ff8-proxy-tls\") pod \"isvc-canary-create-predictor-54d47cbf97-x7vr6\" (UID: \"2cc674bf-c5d9-43c0-99b2-612e8b713ff8\") " pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" Oct 30 22:32:23.626714 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.626700 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2cc674bf-c5d9-43c0-99b2-612e8b713ff8-cabundle-cert\") pod \"isvc-canary-create-predictor-54d47cbf97-x7vr6\" (UID: \"2cc674bf-c5d9-43c0-99b2-612e8b713ff8\") " pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" Oct 30 22:32:23.626775 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.626752 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2cc674bf-c5d9-43c0-99b2-612e8b713ff8-kserve-provision-location\") pod \"isvc-canary-create-predictor-54d47cbf97-x7vr6\" (UID: \"2cc674bf-c5d9-43c0-99b2-612e8b713ff8\") " pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" Oct 30 22:32:23.673832 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.673809 5949 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" Oct 30 22:32:23.727830 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.727803 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2cc674bf-c5d9-43c0-99b2-612e8b713ff8-proxy-tls\") pod \"isvc-canary-create-predictor-54d47cbf97-x7vr6\" (UID: \"2cc674bf-c5d9-43c0-99b2-612e8b713ff8\") " pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" Oct 30 22:32:23.727966 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.727841 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2cc674bf-c5d9-43c0-99b2-612e8b713ff8-cabundle-cert\") pod \"isvc-canary-create-predictor-54d47cbf97-x7vr6\" (UID: \"2cc674bf-c5d9-43c0-99b2-612e8b713ff8\") " pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" Oct 30 22:32:23.727966 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.727864 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2cc674bf-c5d9-43c0-99b2-612e8b713ff8-kserve-provision-location\") pod \"isvc-canary-create-predictor-54d47cbf97-x7vr6\" (UID: \"2cc674bf-c5d9-43c0-99b2-612e8b713ff8\") " pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" Oct 30 22:32:23.727966 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.727893 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njcgt\" (UniqueName: \"kubernetes.io/projected/2cc674bf-c5d9-43c0-99b2-612e8b713ff8-kube-api-access-njcgt\") pod \"isvc-canary-create-predictor-54d47cbf97-x7vr6\" (UID: \"2cc674bf-c5d9-43c0-99b2-612e8b713ff8\") " pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" Oct 30 22:32:23.728138 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.727988 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2cc674bf-c5d9-43c0-99b2-612e8b713ff8-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-predictor-54d47cbf97-x7vr6\" (UID: \"2cc674bf-c5d9-43c0-99b2-612e8b713ff8\") " pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" Oct 30 22:32:23.728634 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.728613 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2cc674bf-c5d9-43c0-99b2-612e8b713ff8-kserve-provision-location\") pod \"isvc-canary-create-predictor-54d47cbf97-x7vr6\" (UID: \"2cc674bf-c5d9-43c0-99b2-612e8b713ff8\") " pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" Oct 30 22:32:23.728807 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.728730 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2cc674bf-c5d9-43c0-99b2-612e8b713ff8-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-predictor-54d47cbf97-x7vr6\" (UID: \"2cc674bf-c5d9-43c0-99b2-612e8b713ff8\") " pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" Oct 30 22:32:23.728882 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.728739 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2cc674bf-c5d9-43c0-99b2-612e8b713ff8-cabundle-cert\") pod \"isvc-canary-create-predictor-54d47cbf97-x7vr6\" (UID: \"2cc674bf-c5d9-43c0-99b2-612e8b713ff8\") " pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" Oct 30 22:32:23.733245 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.730326 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2cc674bf-c5d9-43c0-99b2-612e8b713ff8-proxy-tls\") pod \"isvc-canary-create-predictor-54d47cbf97-x7vr6\" (UID: \"2cc674bf-c5d9-43c0-99b2-612e8b713ff8\") " pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" Oct 30 22:32:23.751772 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.751745 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-njcgt\" (UniqueName: \"kubernetes.io/projected/2cc674bf-c5d9-43c0-99b2-612e8b713ff8-kube-api-access-njcgt\") pod \"isvc-canary-create-predictor-54d47cbf97-x7vr6\" (UID: \"2cc674bf-c5d9-43c0-99b2-612e8b713ff8\") " pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" Oct 30 22:32:23.850908 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.850877 5949 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" Oct 30 22:32:23.975895 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:23.975862 5949 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7"] Oct 30 22:32:23.979051 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:32:23.979028 5949 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18f63ac1_385a_4873_a4e9_49a2ecf97b38.slice/crio-d6b83a3eb1243ef45a049c047923a09897a0b1cd25fee31e45a1ed2aa3b629fa WatchSource:0}: Error finding container d6b83a3eb1243ef45a049c047923a09897a0b1cd25fee31e45a1ed2aa3b629fa: Status 404 returned error can't find the container with id d6b83a3eb1243ef45a049c047923a09897a0b1cd25fee31e45a1ed2aa3b629fa Oct 30 22:32:24.152512 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:24.152432 5949 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6"] Oct 30 22:32:24.156258 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:32:24.156202 5949 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cc674bf_c5d9_43c0_99b2_612e8b713ff8.slice/crio-10fc45bb23607e5efbc1a2ada12a8f3d666cd836e25df76da6f0957728b36163 WatchSource:0}: Error finding container 10fc45bb23607e5efbc1a2ada12a8f3d666cd836e25df76da6f0957728b36163: Status 404 returned error can't find the container with id 10fc45bb23607e5efbc1a2ada12a8f3d666cd836e25df76da6f0957728b36163 Oct 30 22:32:24.663074 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:24.663035 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" event={"ID":"2cc674bf-c5d9-43c0-99b2-612e8b713ff8","Type":"ContainerStarted","Data":"2fff4289555cc62f62538fb92bf73dd9efef09e32d0ee286c608b5c6a9b30ed6"} Oct 30 22:32:24.663502 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:24.663136 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" event={"ID":"2cc674bf-c5d9-43c0-99b2-612e8b713ff8","Type":"ContainerStarted","Data":"10fc45bb23607e5efbc1a2ada12a8f3d666cd836e25df76da6f0957728b36163"} Oct 30 22:32:24.665056 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:24.665022 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" event={"ID":"18f63ac1-385a-4873-a4e9-49a2ecf97b38","Type":"ContainerStarted","Data":"ca86ee4969dea998d9930820452a07fc87a48d2c2eda157361486922ffbc509e"} Oct 30 22:32:24.665166 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:24.665075 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" event={"ID":"18f63ac1-385a-4873-a4e9-49a2ecf97b38","Type":"ContainerStarted","Data":"d6b83a3eb1243ef45a049c047923a09897a0b1cd25fee31e45a1ed2aa3b629fa"} Oct 30 22:32:25.335974 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:32:25.335928 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:32:28.683474 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:28.683438 5949 generic.go:356] "Generic (PLEG): container finished" podID="18f63ac1-385a-4873-a4e9-49a2ecf97b38" containerID="ca86ee4969dea998d9930820452a07fc87a48d2c2eda157361486922ffbc509e" exitCode=0 Oct 30 22:32:28.683948 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:28.683510 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" event={"ID":"18f63ac1-385a-4873-a4e9-49a2ecf97b38","Type":"ContainerDied","Data":"ca86ee4969dea998d9930820452a07fc87a48d2c2eda157361486922ffbc509e"} Oct 30 22:32:28.684999 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:28.684979 5949 generic.go:356] "Generic (PLEG): container finished" podID="2cc674bf-c5d9-43c0-99b2-612e8b713ff8" containerID="2fff4289555cc62f62538fb92bf73dd9efef09e32d0ee286c608b5c6a9b30ed6" exitCode=0 Oct 30 22:32:28.685071 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:28.685041 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" event={"ID":"2cc674bf-c5d9-43c0-99b2-612e8b713ff8","Type":"ContainerDied","Data":"2fff4289555cc62f62538fb92bf73dd9efef09e32d0ee286c608b5c6a9b30ed6"} Oct 30 22:32:29.690829 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:29.690790 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" event={"ID":"2cc674bf-c5d9-43c0-99b2-612e8b713ff8","Type":"ContainerStarted","Data":"01d8eb09d630864714e6d58611a000e966f6b07a594cf4b3b8f7cd7152031c52"} Oct 30 22:32:29.691306 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:29.690898 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" event={"ID":"2cc674bf-c5d9-43c0-99b2-612e8b713ff8","Type":"ContainerStarted","Data":"1bd6ca2ef9f499f2643ba133c6a96bb8836850b68e84913ed884c994d65b71e1"} Oct 30 22:32:29.691306 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:29.691108 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" Oct 30 22:32:29.692829 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:29.692807 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" event={"ID":"18f63ac1-385a-4873-a4e9-49a2ecf97b38","Type":"ContainerStarted","Data":"a006b4daaaa513fbba07347688f39b62f272c516df3716368b3257dd75ee1ba7"} Oct 30 22:32:29.692966 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:29.692861 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" event={"ID":"18f63ac1-385a-4873-a4e9-49a2ecf97b38","Type":"ContainerStarted","Data":"005f65c6d5996afe35401b137291366906c82f58981b18cced71a97eb1f3a84c"} Oct 30 22:32:29.693084 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:29.693065 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" Oct 30 22:32:29.746201 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:29.746157 5949 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" podStartSLOduration=2.910832055 podStartE2EDuration="6.746146975s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.83531492s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:32:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:32:29.745772022 +0000 UTC m=+1437.709580592" watchObservedRunningTime="2026-10-30 22:32:29.746146975 +0000 UTC m=+1437.709955545" Oct 30 22:32:29.793009 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:29.792957 5949 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" podStartSLOduration=2.997860567 podStartE2EDuration="6.792943257s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.79508269s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:32:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:32:29.792367904 +0000 UTC m=+1437.756176473" watchObservedRunningTime="2026-10-30 22:32:29.792943257 +0000 UTC m=+1437.756751821" Oct 30 22:32:30.698090 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:30.698055 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" Oct 30 22:32:30.698559 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:30.698119 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" Oct 30 22:32:30.699030 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:30.699005 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" podUID="18f63ac1-385a-4873-a4e9-49a2ecf97b38" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused" Oct 30 22:32:30.699097 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:30.699023 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" podUID="2cc674bf-c5d9-43c0-99b2-612e8b713ff8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused" Oct 30 22:32:31.701931 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:31.701889 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" podUID="2cc674bf-c5d9-43c0-99b2-612e8b713ff8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused" Oct 30 22:32:31.701931 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:31.701927 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" podUID="18f63ac1-385a-4873-a4e9-49a2ecf97b38" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused" Oct 30 22:32:36.707826 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:36.707795 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" Oct 30 22:32:36.708270 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:36.708061 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" Oct 30 22:32:36.708351 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:36.708302 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" podUID="2cc674bf-c5d9-43c0-99b2-612e8b713ff8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused" Oct 30 22:32:36.708756 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:36.708729 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" podUID="18f63ac1-385a-4873-a4e9-49a2ecf97b38" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused" Oct 30 22:32:46.708293 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:46.708197 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" podUID="2cc674bf-c5d9-43c0-99b2-612e8b713ff8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused" Oct 30 22:32:46.709427 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:46.709397 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" podUID="18f63ac1-385a-4873-a4e9-49a2ecf97b38" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused" Oct 30 22:32:55.336664 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:32:55.336620 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:32:56.709252 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:56.709199 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" podUID="18f63ac1-385a-4873-a4e9-49a2ecf97b38" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused" Oct 30 22:32:56.709638 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:32:56.709200 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" podUID="2cc674bf-c5d9-43c0-99b2-612e8b713ff8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused" Oct 30 22:33:06.708971 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:06.708931 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" podUID="18f63ac1-385a-4873-a4e9-49a2ecf97b38" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused" Oct 30 22:33:06.709443 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:06.708931 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" podUID="2cc674bf-c5d9-43c0-99b2-612e8b713ff8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused" Oct 30 22:33:16.709025 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:16.708982 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" podUID="18f63ac1-385a-4873-a4e9-49a2ecf97b38" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused" Oct 30 22:33:16.709511 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:16.708982 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" podUID="2cc674bf-c5d9-43c0-99b2-612e8b713ff8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused" Oct 30 22:33:25.336025 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:33:25.335984 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:33:26.708322 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:26.708281 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" podUID="2cc674bf-c5d9-43c0-99b2-612e8b713ff8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused" Oct 30 22:33:26.708767 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:26.708743 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" podUID="18f63ac1-385a-4873-a4e9-49a2ecf97b38" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused" Oct 30 22:33:32.297863 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:32.297834 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ql6lh_074eeae7-932c-4642-8fea-671898135ab1/ovn-acl-logging/0.log" Oct 30 22:33:32.302623 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:32.302603 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ql6lh_074eeae7-932c-4642-8fea-671898135ab1/ovn-acl-logging/0.log" Oct 30 22:33:36.708797 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:36.708755 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" podUID="18f63ac1-385a-4873-a4e9-49a2ecf97b38" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused" Oct 30 22:33:36.709180 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:36.708769 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" podUID="2cc674bf-c5d9-43c0-99b2-612e8b713ff8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused" Oct 30 22:33:42.159209 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:42.159166 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" podUID="2cc674bf-c5d9-43c0-99b2-612e8b713ff8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused" Oct 30 22:33:46.709371 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:46.709343 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" Oct 30 22:33:47.229372 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:47.229342 5949 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6"] Oct 30 22:33:47.229587 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:47.229562 5949 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" podUID="2cc674bf-c5d9-43c0-99b2-612e8b713ff8" containerName="kserve-container" containerID="cri-o://1bd6ca2ef9f499f2643ba133c6a96bb8836850b68e84913ed884c994d65b71e1" gracePeriod=30 Oct 30 22:33:47.229660 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:47.229618 5949 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" podUID="2cc674bf-c5d9-43c0-99b2-612e8b713ff8" containerName="kube-rbac-proxy" containerID="cri-o://01d8eb09d630864714e6d58611a000e966f6b07a594cf4b3b8f7cd7152031c52" gracePeriod=30 Oct 30 22:33:47.230695 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:47.230585 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" Oct 30 22:33:47.995280 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:47.995244 5949 generic.go:356] "Generic (PLEG): container finished" podID="2cc674bf-c5d9-43c0-99b2-612e8b713ff8" containerID="01d8eb09d630864714e6d58611a000e966f6b07a594cf4b3b8f7cd7152031c52" exitCode=0 Oct 30 22:33:47.995680 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:47.995252 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" event={"ID":"2cc674bf-c5d9-43c0-99b2-612e8b713ff8","Type":"ContainerDied","Data":"01d8eb09d630864714e6d58611a000e966f6b07a594cf4b3b8f7cd7152031c52"} Oct 30 22:33:51.702387 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:51.702351 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" podUID="2cc674bf-c5d9-43c0-99b2-612e8b713ff8" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.31:8643/healthz\": dial tcp 10.134.0.31:8643: connect: connection refused" Oct 30 22:33:51.872619 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:51.872590 5949 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" Oct 30 22:33:51.989499 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:51.989425 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2cc674bf-c5d9-43c0-99b2-612e8b713ff8-proxy-tls\" (UniqueName: \"kubernetes.io/secret/2cc674bf-c5d9-43c0-99b2-612e8b713ff8-proxy-tls\") pod \"2cc674bf-c5d9-43c0-99b2-612e8b713ff8\" (UID: \"2cc674bf-c5d9-43c0-99b2-612e8b713ff8\") " Oct 30 22:33:51.989499 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:51.989460 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2cc674bf-c5d9-43c0-99b2-612e8b713ff8-kube-api-access-njcgt\" (UniqueName: \"kubernetes.io/projected/2cc674bf-c5d9-43c0-99b2-612e8b713ff8-kube-api-access-njcgt\") pod \"2cc674bf-c5d9-43c0-99b2-612e8b713ff8\" (UID: \"2cc674bf-c5d9-43c0-99b2-612e8b713ff8\") " Oct 30 22:33:51.989714 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:51.989524 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2cc674bf-c5d9-43c0-99b2-612e8b713ff8-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2cc674bf-c5d9-43c0-99b2-612e8b713ff8-kserve-provision-location\") pod \"2cc674bf-c5d9-43c0-99b2-612e8b713ff8\" (UID: \"2cc674bf-c5d9-43c0-99b2-612e8b713ff8\") " Oct 30 22:33:51.989714 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:51.989547 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2cc674bf-c5d9-43c0-99b2-612e8b713ff8-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2cc674bf-c5d9-43c0-99b2-612e8b713ff8-kube-rbac-proxy-sar-config\") pod \"2cc674bf-c5d9-43c0-99b2-612e8b713ff8\" (UID: \"2cc674bf-c5d9-43c0-99b2-612e8b713ff8\") " Oct 30 22:33:51.989714 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:51.989608 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2cc674bf-c5d9-43c0-99b2-612e8b713ff8-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2cc674bf-c5d9-43c0-99b2-612e8b713ff8-cabundle-cert\") pod \"2cc674bf-c5d9-43c0-99b2-612e8b713ff8\" (UID: \"2cc674bf-c5d9-43c0-99b2-612e8b713ff8\") " Oct 30 22:33:51.989945 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:51.989921 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cc674bf-c5d9-43c0-99b2-612e8b713ff8-kserve-provision-location" pod "2cc674bf-c5d9-43c0-99b2-612e8b713ff8" (UID: "2cc674bf-c5d9-43c0-99b2-612e8b713ff8"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 22:33:51.990027 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:51.990000 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2cc674bf-c5d9-43c0-99b2-612e8b713ff8-kube-rbac-proxy-sar-config" pod "2cc674bf-c5d9-43c0-99b2-612e8b713ff8" (UID: "2cc674bf-c5d9-43c0-99b2-612e8b713ff8"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:33:51.990063 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:51.990012 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2cc674bf-c5d9-43c0-99b2-612e8b713ff8-cabundle-cert" pod "2cc674bf-c5d9-43c0-99b2-612e8b713ff8" (UID: "2cc674bf-c5d9-43c0-99b2-612e8b713ff8"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:33:51.991509 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:51.991485 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2cc674bf-c5d9-43c0-99b2-612e8b713ff8-proxy-tls" pod "2cc674bf-c5d9-43c0-99b2-612e8b713ff8" (UID: "2cc674bf-c5d9-43c0-99b2-612e8b713ff8"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:33:51.991620 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:51.991533 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cc674bf-c5d9-43c0-99b2-612e8b713ff8-kube-api-access-njcgt" pod "2cc674bf-c5d9-43c0-99b2-612e8b713ff8" (UID: "2cc674bf-c5d9-43c0-99b2-612e8b713ff8"). InnerVolumeSpecName "kube-api-access-njcgt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 22:33:52.017697 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:52.017672 5949 generic.go:356] "Generic (PLEG): container finished" podID="2cc674bf-c5d9-43c0-99b2-612e8b713ff8" containerID="1bd6ca2ef9f499f2643ba133c6a96bb8836850b68e84913ed884c994d65b71e1" exitCode=0 Oct 30 22:33:52.017841 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:52.017732 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" event={"ID":"2cc674bf-c5d9-43c0-99b2-612e8b713ff8","Type":"ContainerDied","Data":"1bd6ca2ef9f499f2643ba133c6a96bb8836850b68e84913ed884c994d65b71e1"} Oct 30 22:33:52.017841 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:52.017741 5949 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" Oct 30 22:33:52.017841 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:52.017822 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6" event={"ID":"2cc674bf-c5d9-43c0-99b2-612e8b713ff8","Type":"ContainerDied","Data":"10fc45bb23607e5efbc1a2ada12a8f3d666cd836e25df76da6f0957728b36163"} Oct 30 22:33:52.017998 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:52.017860 5949 scope.go:122] "RemoveContainer" containerID="01d8eb09d630864714e6d58611a000e966f6b07a594cf4b3b8f7cd7152031c52" Oct 30 22:33:52.026126 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:52.026110 5949 scope.go:122] "RemoveContainer" containerID="1bd6ca2ef9f499f2643ba133c6a96bb8836850b68e84913ed884c994d65b71e1" Oct 30 22:33:52.033122 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:52.033097 5949 scope.go:122] "RemoveContainer" containerID="2fff4289555cc62f62538fb92bf73dd9efef09e32d0ee286c608b5c6a9b30ed6" Oct 30 22:33:52.040108 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:52.040092 5949 scope.go:122] "RemoveContainer" containerID="01d8eb09d630864714e6d58611a000e966f6b07a594cf4b3b8f7cd7152031c52" Oct 30 22:33:52.040388 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:33:52.040370 5949 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01d8eb09d630864714e6d58611a000e966f6b07a594cf4b3b8f7cd7152031c52\": container with ID starting with 01d8eb09d630864714e6d58611a000e966f6b07a594cf4b3b8f7cd7152031c52 not found: ID does not exist" containerID="01d8eb09d630864714e6d58611a000e966f6b07a594cf4b3b8f7cd7152031c52" Oct 30 22:33:52.040457 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:52.040396 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01d8eb09d630864714e6d58611a000e966f6b07a594cf4b3b8f7cd7152031c52"} err="failed to get container status \"01d8eb09d630864714e6d58611a000e966f6b07a594cf4b3b8f7cd7152031c52\": rpc error: code = NotFound desc = could not find container \"01d8eb09d630864714e6d58611a000e966f6b07a594cf4b3b8f7cd7152031c52\": container with ID starting with 01d8eb09d630864714e6d58611a000e966f6b07a594cf4b3b8f7cd7152031c52 not found: ID does not exist" Oct 30 22:33:52.040457 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:52.040412 5949 scope.go:122] "RemoveContainer" containerID="1bd6ca2ef9f499f2643ba133c6a96bb8836850b68e84913ed884c994d65b71e1" Oct 30 22:33:52.040668 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:33:52.040650 5949 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1bd6ca2ef9f499f2643ba133c6a96bb8836850b68e84913ed884c994d65b71e1\": container with ID starting with 1bd6ca2ef9f499f2643ba133c6a96bb8836850b68e84913ed884c994d65b71e1 not found: ID does not exist" containerID="1bd6ca2ef9f499f2643ba133c6a96bb8836850b68e84913ed884c994d65b71e1" Oct 30 22:33:52.040760 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:52.040673 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bd6ca2ef9f499f2643ba133c6a96bb8836850b68e84913ed884c994d65b71e1"} err="failed to get container status \"1bd6ca2ef9f499f2643ba133c6a96bb8836850b68e84913ed884c994d65b71e1\": rpc error: code = NotFound desc = could not find container \"1bd6ca2ef9f499f2643ba133c6a96bb8836850b68e84913ed884c994d65b71e1\": container with ID starting with 1bd6ca2ef9f499f2643ba133c6a96bb8836850b68e84913ed884c994d65b71e1 not found: ID does not exist" Oct 30 22:33:52.040760 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:52.040694 5949 scope.go:122] "RemoveContainer" containerID="2fff4289555cc62f62538fb92bf73dd9efef09e32d0ee286c608b5c6a9b30ed6" Oct 30 22:33:52.040927 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:33:52.040911 5949 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2fff4289555cc62f62538fb92bf73dd9efef09e32d0ee286c608b5c6a9b30ed6\": container with ID starting with 2fff4289555cc62f62538fb92bf73dd9efef09e32d0ee286c608b5c6a9b30ed6 not found: ID does not exist" containerID="2fff4289555cc62f62538fb92bf73dd9efef09e32d0ee286c608b5c6a9b30ed6" Oct 30 22:33:52.041006 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:52.040931 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fff4289555cc62f62538fb92bf73dd9efef09e32d0ee286c608b5c6a9b30ed6"} err="failed to get container status \"2fff4289555cc62f62538fb92bf73dd9efef09e32d0ee286c608b5c6a9b30ed6\": rpc error: code = NotFound desc = could not find container \"2fff4289555cc62f62538fb92bf73dd9efef09e32d0ee286c608b5c6a9b30ed6\": container with ID starting with 2fff4289555cc62f62538fb92bf73dd9efef09e32d0ee286c608b5c6a9b30ed6 not found: ID does not exist" Oct 30 22:33:52.070385 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:52.070359 5949 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6"] Oct 30 22:33:52.076157 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:52.076139 5949 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-create-predictor-54d47cbf97-x7vr6"] Oct 30 22:33:52.090897 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:52.090876 5949 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2cc674bf-c5d9-43c0-99b2-612e8b713ff8-kserve-provision-location\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:33:52.090897 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:52.090895 5949 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2cc674bf-c5d9-43c0-99b2-612e8b713ff8-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:33:52.091011 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:52.090906 5949 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2cc674bf-c5d9-43c0-99b2-612e8b713ff8-cabundle-cert\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:33:52.091011 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:52.090918 5949 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2cc674bf-c5d9-43c0-99b2-612e8b713ff8-proxy-tls\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:33:52.091011 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:52.090926 5949 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njcgt\" (UniqueName: \"kubernetes.io/projected/2cc674bf-c5d9-43c0-99b2-612e8b713ff8-kube-api-access-njcgt\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:33:52.159793 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:52.159764 5949 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2cc674bf-c5d9-43c0-99b2-612e8b713ff8" path="/var/lib/kubelet/pods/2cc674bf-c5d9-43c0-99b2-612e8b713ff8/volumes" Oct 30 22:33:53.451461 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:53.451430 5949 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7"] Oct 30 22:33:53.451935 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:53.451683 5949 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" podUID="18f63ac1-385a-4873-a4e9-49a2ecf97b38" containerName="kserve-container" containerID="cri-o://005f65c6d5996afe35401b137291366906c82f58981b18cced71a97eb1f3a84c" gracePeriod=30 Oct 30 22:33:53.451935 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:53.451715 5949 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" podUID="18f63ac1-385a-4873-a4e9-49a2ecf97b38" containerName="kube-rbac-proxy" containerID="cri-o://a006b4daaaa513fbba07347688f39b62f272c516df3716368b3257dd75ee1ba7" gracePeriod=30 Oct 30 22:33:54.028441 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:54.028406 5949 generic.go:356] "Generic (PLEG): container finished" podID="18f63ac1-385a-4873-a4e9-49a2ecf97b38" containerID="a006b4daaaa513fbba07347688f39b62f272c516df3716368b3257dd75ee1ba7" exitCode=0 Oct 30 22:33:54.028612 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:54.028466 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" event={"ID":"18f63ac1-385a-4873-a4e9-49a2ecf97b38","Type":"ContainerDied","Data":"a006b4daaaa513fbba07347688f39b62f272c516df3716368b3257dd75ee1ba7"} Oct 30 22:33:55.334896 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:33:55.334865 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:33:56.702956 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:56.702917 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" podUID="18f63ac1-385a-4873-a4e9-49a2ecf97b38" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.30:8643/healthz\": dial tcp 10.134.0.30:8643: connect: connection refused" Oct 30 22:33:56.709436 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:56.709408 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" podUID="18f63ac1-385a-4873-a4e9-49a2ecf97b38" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused" Oct 30 22:33:58.045606 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:58.045575 5949 generic.go:356] "Generic (PLEG): container finished" podID="18f63ac1-385a-4873-a4e9-49a2ecf97b38" containerID="005f65c6d5996afe35401b137291366906c82f58981b18cced71a97eb1f3a84c" exitCode=0 Oct 30 22:33:58.045934 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:58.045644 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" event={"ID":"18f63ac1-385a-4873-a4e9-49a2ecf97b38","Type":"ContainerDied","Data":"005f65c6d5996afe35401b137291366906c82f58981b18cced71a97eb1f3a84c"} Oct 30 22:33:58.097780 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:58.097759 5949 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" Oct 30 22:33:58.241356 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:58.241289 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/18f63ac1-385a-4873-a4e9-49a2ecf97b38-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/18f63ac1-385a-4873-a4e9-49a2ecf97b38-kserve-provision-location\") pod \"18f63ac1-385a-4873-a4e9-49a2ecf97b38\" (UID: \"18f63ac1-385a-4873-a4e9-49a2ecf97b38\") " Oct 30 22:33:58.241484 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:58.241357 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/18f63ac1-385a-4873-a4e9-49a2ecf97b38-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/18f63ac1-385a-4873-a4e9-49a2ecf97b38-cabundle-cert\") pod \"18f63ac1-385a-4873-a4e9-49a2ecf97b38\" (UID: \"18f63ac1-385a-4873-a4e9-49a2ecf97b38\") " Oct 30 22:33:58.241484 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:58.241377 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/18f63ac1-385a-4873-a4e9-49a2ecf97b38-kube-api-access-ctdj4\" (UniqueName: \"kubernetes.io/projected/18f63ac1-385a-4873-a4e9-49a2ecf97b38-kube-api-access-ctdj4\") pod \"18f63ac1-385a-4873-a4e9-49a2ecf97b38\" (UID: \"18f63ac1-385a-4873-a4e9-49a2ecf97b38\") " Oct 30 22:33:58.241484 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:58.241440 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/18f63ac1-385a-4873-a4e9-49a2ecf97b38-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/18f63ac1-385a-4873-a4e9-49a2ecf97b38-kube-rbac-proxy-sar-config\") pod \"18f63ac1-385a-4873-a4e9-49a2ecf97b38\" (UID: \"18f63ac1-385a-4873-a4e9-49a2ecf97b38\") " Oct 30 22:33:58.241636 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:58.241486 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/18f63ac1-385a-4873-a4e9-49a2ecf97b38-proxy-tls\" (UniqueName: \"kubernetes.io/secret/18f63ac1-385a-4873-a4e9-49a2ecf97b38-proxy-tls\") pod \"18f63ac1-385a-4873-a4e9-49a2ecf97b38\" (UID: \"18f63ac1-385a-4873-a4e9-49a2ecf97b38\") " Oct 30 22:33:58.241636 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:58.241610 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18f63ac1-385a-4873-a4e9-49a2ecf97b38-kserve-provision-location" pod "18f63ac1-385a-4873-a4e9-49a2ecf97b38" (UID: "18f63ac1-385a-4873-a4e9-49a2ecf97b38"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 22:33:58.241743 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:58.241721 5949 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/18f63ac1-385a-4873-a4e9-49a2ecf97b38-kserve-provision-location\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:33:58.241806 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:58.241789 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18f63ac1-385a-4873-a4e9-49a2ecf97b38-cabundle-cert" pod "18f63ac1-385a-4873-a4e9-49a2ecf97b38" (UID: "18f63ac1-385a-4873-a4e9-49a2ecf97b38"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:33:58.241895 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:58.241872 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18f63ac1-385a-4873-a4e9-49a2ecf97b38-kube-rbac-proxy-sar-config" pod "18f63ac1-385a-4873-a4e9-49a2ecf97b38" (UID: "18f63ac1-385a-4873-a4e9-49a2ecf97b38"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:33:58.243473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:58.243451 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18f63ac1-385a-4873-a4e9-49a2ecf97b38-kube-api-access-ctdj4" pod "18f63ac1-385a-4873-a4e9-49a2ecf97b38" (UID: "18f63ac1-385a-4873-a4e9-49a2ecf97b38"). InnerVolumeSpecName "kube-api-access-ctdj4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 22:33:58.243613 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:58.243592 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18f63ac1-385a-4873-a4e9-49a2ecf97b38-proxy-tls" pod "18f63ac1-385a-4873-a4e9-49a2ecf97b38" (UID: "18f63ac1-385a-4873-a4e9-49a2ecf97b38"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:33:58.342164 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:58.342136 5949 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/18f63ac1-385a-4873-a4e9-49a2ecf97b38-cabundle-cert\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:33:58.342164 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:58.342159 5949 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ctdj4\" (UniqueName: \"kubernetes.io/projected/18f63ac1-385a-4873-a4e9-49a2ecf97b38-kube-api-access-ctdj4\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:33:58.342164 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:58.342169 5949 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/18f63ac1-385a-4873-a4e9-49a2ecf97b38-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:33:58.342368 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:58.342178 5949 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/18f63ac1-385a-4873-a4e9-49a2ecf97b38-proxy-tls\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:33:59.051101 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:59.051068 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" event={"ID":"18f63ac1-385a-4873-a4e9-49a2ecf97b38","Type":"ContainerDied","Data":"d6b83a3eb1243ef45a049c047923a09897a0b1cd25fee31e45a1ed2aa3b629fa"} Oct 30 22:33:59.051573 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:59.051120 5949 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7" Oct 30 22:33:59.051573 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:59.051135 5949 scope.go:122] "RemoveContainer" containerID="a006b4daaaa513fbba07347688f39b62f272c516df3716368b3257dd75ee1ba7" Oct 30 22:33:59.059790 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:59.059774 5949 scope.go:122] "RemoveContainer" containerID="005f65c6d5996afe35401b137291366906c82f58981b18cced71a97eb1f3a84c" Oct 30 22:33:59.067012 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:59.066997 5949 scope.go:122] "RemoveContainer" containerID="ca86ee4969dea998d9930820452a07fc87a48d2c2eda157361486922ffbc509e" Oct 30 22:33:59.112520 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:59.112492 5949 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7"] Oct 30 22:33:59.124306 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:33:59.124287 5949 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-create-v2-predictor-86bcf9785b-mwzq7"] Oct 30 22:34:00.159354 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:00.159323 5949 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="18f63ac1-385a-4873-a4e9-49a2ecf97b38" path="/var/lib/kubelet/pods/18f63ac1-385a-4873-a4e9-49a2ecf97b38/volumes" Oct 30 22:34:01.295327 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:01.295293 5949 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv"] Oct 30 22:34:01.295697 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:01.295622 5949 cpu_manager.go:414] "containerMap: removing container" podUID="2cc674bf-c5d9-43c0-99b2-612e8b713ff8" containerName="kube-rbac-proxy" Oct 30 22:34:01.295697 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:01.295637 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2cc674bf-c5d9-43c0-99b2-612e8b713ff8" containerName="kube-rbac-proxy" Oct 30 22:34:01.295697 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:01.295657 5949 cpu_manager.go:414] "containerMap: removing container" podUID="2cc674bf-c5d9-43c0-99b2-612e8b713ff8" containerName="storage-initializer" Oct 30 22:34:01.295697 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:01.295662 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2cc674bf-c5d9-43c0-99b2-612e8b713ff8" containerName="storage-initializer" Oct 30 22:34:01.295697 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:01.295674 5949 cpu_manager.go:414] "containerMap: removing container" podUID="18f63ac1-385a-4873-a4e9-49a2ecf97b38" containerName="storage-initializer" Oct 30 22:34:01.295697 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:01.295679 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="18f63ac1-385a-4873-a4e9-49a2ecf97b38" containerName="storage-initializer" Oct 30 22:34:01.295697 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:01.295689 5949 cpu_manager.go:414] "containerMap: removing container" podUID="2cc674bf-c5d9-43c0-99b2-612e8b713ff8" containerName="kserve-container" Oct 30 22:34:01.295697 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:01.295694 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2cc674bf-c5d9-43c0-99b2-612e8b713ff8" containerName="kserve-container" Oct 30 22:34:01.295938 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:01.295700 5949 cpu_manager.go:414] "containerMap: removing container" podUID="18f63ac1-385a-4873-a4e9-49a2ecf97b38" containerName="kube-rbac-proxy" Oct 30 22:34:01.295938 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:01.295727 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="18f63ac1-385a-4873-a4e9-49a2ecf97b38" containerName="kube-rbac-proxy" Oct 30 22:34:01.295938 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:01.295734 5949 cpu_manager.go:414] "containerMap: removing container" podUID="18f63ac1-385a-4873-a4e9-49a2ecf97b38" containerName="kserve-container" Oct 30 22:34:01.295938 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:01.295739 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="18f63ac1-385a-4873-a4e9-49a2ecf97b38" containerName="kserve-container" Oct 30 22:34:01.295938 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:01.295787 5949 memory_manager.go:365] "RemoveStaleState removing state" podUID="2cc674bf-c5d9-43c0-99b2-612e8b713ff8" containerName="kserve-container" Oct 30 22:34:01.295938 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:01.295798 5949 memory_manager.go:365] "RemoveStaleState removing state" podUID="18f63ac1-385a-4873-a4e9-49a2ecf97b38" containerName="kube-rbac-proxy" Oct 30 22:34:01.295938 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:01.295804 5949 memory_manager.go:365] "RemoveStaleState removing state" podUID="18f63ac1-385a-4873-a4e9-49a2ecf97b38" containerName="kserve-container" Oct 30 22:34:01.295938 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:01.295812 5949 memory_manager.go:365] "RemoveStaleState removing state" podUID="2cc674bf-c5d9-43c0-99b2-612e8b713ff8" containerName="kube-rbac-proxy" Oct 30 22:34:01.300567 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:01.300547 5949 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" Oct 30 22:34:01.333869 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:01.333834 5949 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv"] Oct 30 22:34:01.364493 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:01.364464 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/628dbe55-e888-4ce3-93a1-d16e69103d91-cabundle-cert\") pod \"isvc-canary-promote-predictor-5dc89bc9c5-nghfv\" (UID: \"628dbe55-e888-4ce3-93a1-d16e69103d91\") " pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" Oct 30 22:34:01.364662 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:01.364500 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/628dbe55-e888-4ce3-93a1-d16e69103d91-proxy-tls\") pod \"isvc-canary-promote-predictor-5dc89bc9c5-nghfv\" (UID: \"628dbe55-e888-4ce3-93a1-d16e69103d91\") " pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" Oct 30 22:34:01.364662 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:01.364580 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/628dbe55-e888-4ce3-93a1-d16e69103d91-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-predictor-5dc89bc9c5-nghfv\" (UID: \"628dbe55-e888-4ce3-93a1-d16e69103d91\") " pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" Oct 30 22:34:01.364662 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:01.364602 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/628dbe55-e888-4ce3-93a1-d16e69103d91-kserve-provision-location\") pod \"isvc-canary-promote-predictor-5dc89bc9c5-nghfv\" (UID: \"628dbe55-e888-4ce3-93a1-d16e69103d91\") " pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" Oct 30 22:34:01.364662 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:01.364624 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5zg6\" (UniqueName: \"kubernetes.io/projected/628dbe55-e888-4ce3-93a1-d16e69103d91-kube-api-access-h5zg6\") pod \"isvc-canary-promote-predictor-5dc89bc9c5-nghfv\" (UID: \"628dbe55-e888-4ce3-93a1-d16e69103d91\") " pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" Oct 30 22:34:01.375907 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:01.375876 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-306b77ff42\"/\"seaweedfs-s3-creds\"" Oct 30 22:34:01.375907 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:01.375879 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-306b77ff42\"/\"isvc-canary-promote-predictor-serving-cert\"" Oct 30 22:34:01.376055 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:01.375925 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-306b77ff42\"/\"default-dockercfg-7rhfl\"" Oct 30 22:34:01.381161 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:01.381146 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-306b77ff42\"/\"odh-kserve-custom-ca-bundle\"" Oct 30 22:34:01.381277 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:01.381263 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-306b77ff42\"/\"isvc-canary-promote-kube-rbac-proxy-sar-config\"" Oct 30 22:34:01.381321 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:01.381268 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-306b77ff42\"/\"openshift-service-ca.crt\"" Oct 30 22:34:01.381353 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:01.381319 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-306b77ff42\"/\"kube-root-ca.crt\"" Oct 30 22:34:01.465362 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:01.465329 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/628dbe55-e888-4ce3-93a1-d16e69103d91-cabundle-cert\") pod \"isvc-canary-promote-predictor-5dc89bc9c5-nghfv\" (UID: \"628dbe55-e888-4ce3-93a1-d16e69103d91\") " pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" Oct 30 22:34:01.465362 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:01.465369 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/628dbe55-e888-4ce3-93a1-d16e69103d91-proxy-tls\") pod \"isvc-canary-promote-predictor-5dc89bc9c5-nghfv\" (UID: \"628dbe55-e888-4ce3-93a1-d16e69103d91\") " pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" Oct 30 22:34:01.465565 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:01.465401 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/628dbe55-e888-4ce3-93a1-d16e69103d91-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-predictor-5dc89bc9c5-nghfv\" (UID: \"628dbe55-e888-4ce3-93a1-d16e69103d91\") " pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" Oct 30 22:34:01.465565 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:01.465435 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/628dbe55-e888-4ce3-93a1-d16e69103d91-kserve-provision-location\") pod \"isvc-canary-promote-predictor-5dc89bc9c5-nghfv\" (UID: \"628dbe55-e888-4ce3-93a1-d16e69103d91\") " pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" Oct 30 22:34:01.465565 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:01.465462 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5zg6\" (UniqueName: \"kubernetes.io/projected/628dbe55-e888-4ce3-93a1-d16e69103d91-kube-api-access-h5zg6\") pod \"isvc-canary-promote-predictor-5dc89bc9c5-nghfv\" (UID: \"628dbe55-e888-4ce3-93a1-d16e69103d91\") " pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" Oct 30 22:34:01.465912 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:01.465891 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/628dbe55-e888-4ce3-93a1-d16e69103d91-kserve-provision-location\") pod \"isvc-canary-promote-predictor-5dc89bc9c5-nghfv\" (UID: \"628dbe55-e888-4ce3-93a1-d16e69103d91\") " pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" Oct 30 22:34:01.466072 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:01.466052 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/628dbe55-e888-4ce3-93a1-d16e69103d91-cabundle-cert\") pod \"isvc-canary-promote-predictor-5dc89bc9c5-nghfv\" (UID: \"628dbe55-e888-4ce3-93a1-d16e69103d91\") " pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" Oct 30 22:34:01.466143 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:01.466093 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/628dbe55-e888-4ce3-93a1-d16e69103d91-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-predictor-5dc89bc9c5-nghfv\" (UID: \"628dbe55-e888-4ce3-93a1-d16e69103d91\") " pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" Oct 30 22:34:01.467845 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:01.467823 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/628dbe55-e888-4ce3-93a1-d16e69103d91-proxy-tls\") pod \"isvc-canary-promote-predictor-5dc89bc9c5-nghfv\" (UID: \"628dbe55-e888-4ce3-93a1-d16e69103d91\") " pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" Oct 30 22:34:01.490340 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:01.490311 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5zg6\" (UniqueName: \"kubernetes.io/projected/628dbe55-e888-4ce3-93a1-d16e69103d91-kube-api-access-h5zg6\") pod \"isvc-canary-promote-predictor-5dc89bc9c5-nghfv\" (UID: \"628dbe55-e888-4ce3-93a1-d16e69103d91\") " pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" Oct 30 22:34:01.616941 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:01.616843 5949 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" Oct 30 22:34:01.922953 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:01.922925 5949 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv"] Oct 30 22:34:01.926521 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:34:01.926497 5949 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod628dbe55_e888_4ce3_93a1_d16e69103d91.slice/crio-8ec9b765709962a1eaca5a92ac939c1e049be6469cd5a59c242a841148fedbd2 WatchSource:0}: Error finding container 8ec9b765709962a1eaca5a92ac939c1e049be6469cd5a59c242a841148fedbd2: Status 404 returned error can't find the container with id 8ec9b765709962a1eaca5a92ac939c1e049be6469cd5a59c242a841148fedbd2 Oct 30 22:34:02.064976 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:02.064938 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" event={"ID":"628dbe55-e888-4ce3-93a1-d16e69103d91","Type":"ContainerStarted","Data":"affb2ef3b57b63af00b750977c7042d911715d5177f72c69b8055b1b49ac34dc"} Oct 30 22:34:02.065122 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:02.065007 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" event={"ID":"628dbe55-e888-4ce3-93a1-d16e69103d91","Type":"ContainerStarted","Data":"8ec9b765709962a1eaca5a92ac939c1e049be6469cd5a59c242a841148fedbd2"} Oct 30 22:34:06.082196 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:06.082117 5949 generic.go:356] "Generic (PLEG): container finished" podID="628dbe55-e888-4ce3-93a1-d16e69103d91" containerID="affb2ef3b57b63af00b750977c7042d911715d5177f72c69b8055b1b49ac34dc" exitCode=0 Oct 30 22:34:06.082196 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:06.082152 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" event={"ID":"628dbe55-e888-4ce3-93a1-d16e69103d91","Type":"ContainerDied","Data":"affb2ef3b57b63af00b750977c7042d911715d5177f72c69b8055b1b49ac34dc"} Oct 30 22:34:07.088236 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:07.088182 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" event={"ID":"628dbe55-e888-4ce3-93a1-d16e69103d91","Type":"ContainerStarted","Data":"39df165de633fcd9acf6a5694ddb0c4585721cc5dc2d8ce9c8a8493bc9c18d22"} Oct 30 22:34:07.088723 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:07.088276 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" event={"ID":"628dbe55-e888-4ce3-93a1-d16e69103d91","Type":"ContainerStarted","Data":"e2099fec52f979b94126df4e9a53df410697937dfba098ca92687e6848bcbcec"} Oct 30 22:34:07.088723 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:07.088562 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" Oct 30 22:34:07.088723 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:07.088617 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" Oct 30 22:34:07.089902 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:07.089880 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" podUID="628dbe55-e888-4ce3-93a1-d16e69103d91" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused" Oct 30 22:34:07.144952 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:07.144903 5949 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" podStartSLOduration=2.371759521 podStartE2EDuration="6.144891019s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.773131498s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:34:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:34:07.144290407 +0000 UTC m=+1535.108098978" watchObservedRunningTime="2026-10-30 22:34:07.144891019 +0000 UTC m=+1535.108699589" Oct 30 22:34:08.091922 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:08.091878 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" podUID="628dbe55-e888-4ce3-93a1-d16e69103d91" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused" Oct 30 22:34:13.096625 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:13.096596 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" Oct 30 22:34:13.097124 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:13.097098 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" podUID="628dbe55-e888-4ce3-93a1-d16e69103d91" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused" Oct 30 22:34:23.098033 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:23.097992 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" podUID="628dbe55-e888-4ce3-93a1-d16e69103d91" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused" Oct 30 22:34:25.335387 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:34:25.335353 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:34:33.097494 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:33.097455 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" podUID="628dbe55-e888-4ce3-93a1-d16e69103d91" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused" Oct 30 22:34:43.097173 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:43.097130 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" podUID="628dbe55-e888-4ce3-93a1-d16e69103d91" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused" Oct 30 22:34:53.097115 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:34:53.097073 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" podUID="628dbe55-e888-4ce3-93a1-d16e69103d91" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused" Oct 30 22:34:55.334520 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:34:55.334486 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:35:03.097037 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:35:03.096994 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" podUID="628dbe55-e888-4ce3-93a1-d16e69103d91" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused" Oct 30 22:35:13.097407 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:35:13.097367 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" podUID="628dbe55-e888-4ce3-93a1-d16e69103d91" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused" Oct 30 22:35:23.101960 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:35:23.101930 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" Oct 30 22:35:25.335375 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:35:25.335331 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:35:31.380944 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:35:31.380910 5949 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv"] Oct 30 22:35:31.381409 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:35:31.381147 5949 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" podUID="628dbe55-e888-4ce3-93a1-d16e69103d91" containerName="kserve-container" containerID="cri-o://e2099fec52f979b94126df4e9a53df410697937dfba098ca92687e6848bcbcec" gracePeriod=30 Oct 30 22:35:31.381409 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:35:31.381191 5949 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" podUID="628dbe55-e888-4ce3-93a1-d16e69103d91" containerName="kube-rbac-proxy" containerID="cri-o://39df165de633fcd9acf6a5694ddb0c4585721cc5dc2d8ce9c8a8493bc9c18d22" gracePeriod=30 Oct 30 22:35:32.415714 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:35:32.415680 5949 generic.go:356] "Generic (PLEG): container finished" podID="628dbe55-e888-4ce3-93a1-d16e69103d91" containerID="39df165de633fcd9acf6a5694ddb0c4585721cc5dc2d8ce9c8a8493bc9c18d22" exitCode=0 Oct 30 22:35:32.416061 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:35:32.415756 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" event={"ID":"628dbe55-e888-4ce3-93a1-d16e69103d91","Type":"ContainerDied","Data":"39df165de633fcd9acf6a5694ddb0c4585721cc5dc2d8ce9c8a8493bc9c18d22"} Oct 30 22:35:33.092119 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:35:33.092073 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" podUID="628dbe55-e888-4ce3-93a1-d16e69103d91" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.32:8643/healthz\": dial tcp 10.134.0.32:8643: connect: connection refused" Oct 30 22:35:33.097424 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:35:33.097395 5949 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" podUID="628dbe55-e888-4ce3-93a1-d16e69103d91" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused" Oct 30 22:35:36.118958 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:35:36.118932 5949 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" Oct 30 22:35:36.265354 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:35:36.265267 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/628dbe55-e888-4ce3-93a1-d16e69103d91-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/628dbe55-e888-4ce3-93a1-d16e69103d91-cabundle-cert\") pod \"628dbe55-e888-4ce3-93a1-d16e69103d91\" (UID: \"628dbe55-e888-4ce3-93a1-d16e69103d91\") " Oct 30 22:35:36.265354 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:35:36.265315 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/628dbe55-e888-4ce3-93a1-d16e69103d91-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/628dbe55-e888-4ce3-93a1-d16e69103d91-kserve-provision-location\") pod \"628dbe55-e888-4ce3-93a1-d16e69103d91\" (UID: \"628dbe55-e888-4ce3-93a1-d16e69103d91\") " Oct 30 22:35:36.265354 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:35:36.265352 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/628dbe55-e888-4ce3-93a1-d16e69103d91-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/628dbe55-e888-4ce3-93a1-d16e69103d91-kube-rbac-proxy-sar-config\") pod \"628dbe55-e888-4ce3-93a1-d16e69103d91\" (UID: \"628dbe55-e888-4ce3-93a1-d16e69103d91\") " Oct 30 22:35:36.265646 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:35:36.265373 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/628dbe55-e888-4ce3-93a1-d16e69103d91-kube-api-access-h5zg6\" (UniqueName: \"kubernetes.io/projected/628dbe55-e888-4ce3-93a1-d16e69103d91-kube-api-access-h5zg6\") pod \"628dbe55-e888-4ce3-93a1-d16e69103d91\" (UID: \"628dbe55-e888-4ce3-93a1-d16e69103d91\") " Oct 30 22:35:36.265646 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:35:36.265424 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/628dbe55-e888-4ce3-93a1-d16e69103d91-proxy-tls\" (UniqueName: \"kubernetes.io/secret/628dbe55-e888-4ce3-93a1-d16e69103d91-proxy-tls\") pod \"628dbe55-e888-4ce3-93a1-d16e69103d91\" (UID: \"628dbe55-e888-4ce3-93a1-d16e69103d91\") " Oct 30 22:35:36.265752 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:35:36.265661 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/628dbe55-e888-4ce3-93a1-d16e69103d91-kserve-provision-location" pod "628dbe55-e888-4ce3-93a1-d16e69103d91" (UID: "628dbe55-e888-4ce3-93a1-d16e69103d91"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 22:35:36.265752 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:35:36.265695 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/628dbe55-e888-4ce3-93a1-d16e69103d91-cabundle-cert" pod "628dbe55-e888-4ce3-93a1-d16e69103d91" (UID: "628dbe55-e888-4ce3-93a1-d16e69103d91"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:35:36.265856 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:35:36.265777 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/628dbe55-e888-4ce3-93a1-d16e69103d91-kube-rbac-proxy-sar-config" pod "628dbe55-e888-4ce3-93a1-d16e69103d91" (UID: "628dbe55-e888-4ce3-93a1-d16e69103d91"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:35:36.267508 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:35:36.267478 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/628dbe55-e888-4ce3-93a1-d16e69103d91-kube-api-access-h5zg6" pod "628dbe55-e888-4ce3-93a1-d16e69103d91" (UID: "628dbe55-e888-4ce3-93a1-d16e69103d91"). InnerVolumeSpecName "kube-api-access-h5zg6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 22:35:36.267611 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:35:36.267558 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/628dbe55-e888-4ce3-93a1-d16e69103d91-proxy-tls" pod "628dbe55-e888-4ce3-93a1-d16e69103d91" (UID: "628dbe55-e888-4ce3-93a1-d16e69103d91"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:35:36.366430 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:35:36.366402 5949 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/628dbe55-e888-4ce3-93a1-d16e69103d91-cabundle-cert\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:35:36.366430 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:35:36.366427 5949 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/628dbe55-e888-4ce3-93a1-d16e69103d91-kserve-provision-location\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:35:36.366586 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:35:36.366441 5949 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/628dbe55-e888-4ce3-93a1-d16e69103d91-kube-rbac-proxy-sar-config\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:35:36.366586 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:35:36.366453 5949 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5zg6\" (UniqueName: \"kubernetes.io/projected/628dbe55-e888-4ce3-93a1-d16e69103d91-kube-api-access-h5zg6\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:35:36.366586 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:35:36.366464 5949 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/628dbe55-e888-4ce3-93a1-d16e69103d91-proxy-tls\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:35:36.432097 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:35:36.432064 5949 generic.go:356] "Generic (PLEG): container finished" podID="628dbe55-e888-4ce3-93a1-d16e69103d91" containerID="e2099fec52f979b94126df4e9a53df410697937dfba098ca92687e6848bcbcec" exitCode=0 Oct 30 22:35:36.432229 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:35:36.432108 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" event={"ID":"628dbe55-e888-4ce3-93a1-d16e69103d91","Type":"ContainerDied","Data":"e2099fec52f979b94126df4e9a53df410697937dfba098ca92687e6848bcbcec"} Oct 30 22:35:36.432229 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:35:36.432134 5949 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" Oct 30 22:35:36.432229 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:35:36.432155 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv" event={"ID":"628dbe55-e888-4ce3-93a1-d16e69103d91","Type":"ContainerDied","Data":"8ec9b765709962a1eaca5a92ac939c1e049be6469cd5a59c242a841148fedbd2"} Oct 30 22:35:36.432229 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:35:36.432179 5949 scope.go:122] "RemoveContainer" containerID="39df165de633fcd9acf6a5694ddb0c4585721cc5dc2d8ce9c8a8493bc9c18d22" Oct 30 22:35:36.440473 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:35:36.440458 5949 scope.go:122] "RemoveContainer" containerID="e2099fec52f979b94126df4e9a53df410697937dfba098ca92687e6848bcbcec" Oct 30 22:35:36.447848 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:35:36.447824 5949 scope.go:122] "RemoveContainer" containerID="affb2ef3b57b63af00b750977c7042d911715d5177f72c69b8055b1b49ac34dc" Oct 30 22:35:36.454878 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:35:36.454852 5949 scope.go:122] "RemoveContainer" containerID="39df165de633fcd9acf6a5694ddb0c4585721cc5dc2d8ce9c8a8493bc9c18d22" Oct 30 22:35:36.455137 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:35:36.455116 5949 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39df165de633fcd9acf6a5694ddb0c4585721cc5dc2d8ce9c8a8493bc9c18d22\": container with ID starting with 39df165de633fcd9acf6a5694ddb0c4585721cc5dc2d8ce9c8a8493bc9c18d22 not found: ID does not exist" containerID="39df165de633fcd9acf6a5694ddb0c4585721cc5dc2d8ce9c8a8493bc9c18d22" Oct 30 22:35:36.455229 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:35:36.455148 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39df165de633fcd9acf6a5694ddb0c4585721cc5dc2d8ce9c8a8493bc9c18d22"} err="failed to get container status \"39df165de633fcd9acf6a5694ddb0c4585721cc5dc2d8ce9c8a8493bc9c18d22\": rpc error: code = NotFound desc = could not find container \"39df165de633fcd9acf6a5694ddb0c4585721cc5dc2d8ce9c8a8493bc9c18d22\": container with ID starting with 39df165de633fcd9acf6a5694ddb0c4585721cc5dc2d8ce9c8a8493bc9c18d22 not found: ID does not exist" Oct 30 22:35:36.455229 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:35:36.455171 5949 scope.go:122] "RemoveContainer" containerID="e2099fec52f979b94126df4e9a53df410697937dfba098ca92687e6848bcbcec" Oct 30 22:35:36.455427 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:35:36.455408 5949 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2099fec52f979b94126df4e9a53df410697937dfba098ca92687e6848bcbcec\": container with ID starting with e2099fec52f979b94126df4e9a53df410697937dfba098ca92687e6848bcbcec not found: ID does not exist" containerID="e2099fec52f979b94126df4e9a53df410697937dfba098ca92687e6848bcbcec" Oct 30 22:35:36.455466 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:35:36.455434 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2099fec52f979b94126df4e9a53df410697937dfba098ca92687e6848bcbcec"} err="failed to get container status \"e2099fec52f979b94126df4e9a53df410697937dfba098ca92687e6848bcbcec\": rpc error: code = NotFound desc = could not find container \"e2099fec52f979b94126df4e9a53df410697937dfba098ca92687e6848bcbcec\": container with ID starting with e2099fec52f979b94126df4e9a53df410697937dfba098ca92687e6848bcbcec not found: ID does not exist" Oct 30 22:35:36.455466 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:35:36.455449 5949 scope.go:122] "RemoveContainer" containerID="affb2ef3b57b63af00b750977c7042d911715d5177f72c69b8055b1b49ac34dc" Oct 30 22:35:36.455687 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:35:36.455672 5949 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"affb2ef3b57b63af00b750977c7042d911715d5177f72c69b8055b1b49ac34dc\": container with ID starting with affb2ef3b57b63af00b750977c7042d911715d5177f72c69b8055b1b49ac34dc not found: ID does not exist" containerID="affb2ef3b57b63af00b750977c7042d911715d5177f72c69b8055b1b49ac34dc" Oct 30 22:35:36.455739 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:35:36.455695 5949 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"affb2ef3b57b63af00b750977c7042d911715d5177f72c69b8055b1b49ac34dc"} err="failed to get container status \"affb2ef3b57b63af00b750977c7042d911715d5177f72c69b8055b1b49ac34dc\": rpc error: code = NotFound desc = could not find container \"affb2ef3b57b63af00b750977c7042d911715d5177f72c69b8055b1b49ac34dc\": container with ID starting with affb2ef3b57b63af00b750977c7042d911715d5177f72c69b8055b1b49ac34dc not found: ID does not exist" Oct 30 22:35:36.482369 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:35:36.482348 5949 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv"] Oct 30 22:35:36.487180 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:35:36.487158 5949 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-306b77ff42/isvc-canary-promote-predictor-5dc89bc9c5-nghfv"] Oct 30 22:35:38.159835 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:35:38.159802 5949 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="628dbe55-e888-4ce3-93a1-d16e69103d91" path="/var/lib/kubelet/pods/628dbe55-e888-4ce3-93a1-d16e69103d91/volumes" Oct 30 22:35:55.334871 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:35:55.334795 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:36:25.336010 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:36:25.335971 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:36:55.336293 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:36:55.336254 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:37:25.335615 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:37:25.335568 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:37:55.335435 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:37:55.335393 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:38:25.336112 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:38:25.336079 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:38:32.323238 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:38:32.323200 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ql6lh_074eeae7-932c-4642-8fea-671898135ab1/ovn-acl-logging/0.log" Oct 30 22:38:32.328816 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:38:32.328796 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ql6lh_074eeae7-932c-4642-8fea-671898135ab1/ovn-acl-logging/0.log" Oct 30 22:38:55.336077 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:38:55.335999 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:39:25.335133 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:39:25.335099 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:39:48.766105 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:39:48.766064 5949 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bpq5l/must-gather-xxpr6"] Oct 30 22:39:48.766618 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:39:48.766548 5949 cpu_manager.go:414] "containerMap: removing container" podUID="628dbe55-e888-4ce3-93a1-d16e69103d91" containerName="kube-rbac-proxy" Oct 30 22:39:48.766618 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:39:48.766566 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="628dbe55-e888-4ce3-93a1-d16e69103d91" containerName="kube-rbac-proxy" Oct 30 22:39:48.766618 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:39:48.766594 5949 cpu_manager.go:414] "containerMap: removing container" podUID="628dbe55-e888-4ce3-93a1-d16e69103d91" containerName="kserve-container" Oct 30 22:39:48.766618 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:39:48.766601 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="628dbe55-e888-4ce3-93a1-d16e69103d91" containerName="kserve-container" Oct 30 22:39:48.766834 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:39:48.766627 5949 cpu_manager.go:414] "containerMap: removing container" podUID="628dbe55-e888-4ce3-93a1-d16e69103d91" containerName="storage-initializer" Oct 30 22:39:48.766834 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:39:48.766635 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="628dbe55-e888-4ce3-93a1-d16e69103d91" containerName="storage-initializer" Oct 30 22:39:48.766834 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:39:48.766704 5949 memory_manager.go:365] "RemoveStaleState removing state" podUID="628dbe55-e888-4ce3-93a1-d16e69103d91" containerName="kserve-container" Oct 30 22:39:48.766834 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:39:48.766724 5949 memory_manager.go:365] "RemoveStaleState removing state" podUID="628dbe55-e888-4ce3-93a1-d16e69103d91" containerName="kube-rbac-proxy" Oct 30 22:39:48.770088 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:39:48.770065 5949 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bpq5l/must-gather-xxpr6" Oct 30 22:39:48.795547 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:39:48.795513 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bpq5l\"/\"openshift-service-ca.crt\"" Oct 30 22:39:48.795547 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:39:48.795524 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-bpq5l\"/\"default-dockercfg-trwjg\"" Oct 30 22:39:48.795736 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:39:48.795548 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bpq5l\"/\"kube-root-ca.crt\"" Oct 30 22:39:48.796470 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:39:48.796450 5949 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bpq5l/must-gather-xxpr6"] Oct 30 22:39:48.843510 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:39:48.843479 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9943309a-ce74-4888-a855-1e3b01d24f15-must-gather-output\") pod \"must-gather-xxpr6\" (UID: \"9943309a-ce74-4888-a855-1e3b01d24f15\") " pod="openshift-must-gather-bpq5l/must-gather-xxpr6" Oct 30 22:39:48.843611 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:39:48.843520 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4z4r7\" (UniqueName: \"kubernetes.io/projected/9943309a-ce74-4888-a855-1e3b01d24f15-kube-api-access-4z4r7\") pod \"must-gather-xxpr6\" (UID: \"9943309a-ce74-4888-a855-1e3b01d24f15\") " pod="openshift-must-gather-bpq5l/must-gather-xxpr6" Oct 30 22:39:48.944753 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:39:48.944723 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9943309a-ce74-4888-a855-1e3b01d24f15-must-gather-output\") pod \"must-gather-xxpr6\" (UID: \"9943309a-ce74-4888-a855-1e3b01d24f15\") " pod="openshift-must-gather-bpq5l/must-gather-xxpr6" Oct 30 22:39:48.944868 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:39:48.944763 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z4r7\" (UniqueName: \"kubernetes.io/projected/9943309a-ce74-4888-a855-1e3b01d24f15-kube-api-access-4z4r7\") pod \"must-gather-xxpr6\" (UID: \"9943309a-ce74-4888-a855-1e3b01d24f15\") " pod="openshift-must-gather-bpq5l/must-gather-xxpr6" Oct 30 22:39:48.945071 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:39:48.945051 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9943309a-ce74-4888-a855-1e3b01d24f15-must-gather-output\") pod \"must-gather-xxpr6\" (UID: \"9943309a-ce74-4888-a855-1e3b01d24f15\") " pod="openshift-must-gather-bpq5l/must-gather-xxpr6" Oct 30 22:39:48.967493 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:39:48.967462 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4z4r7\" (UniqueName: \"kubernetes.io/projected/9943309a-ce74-4888-a855-1e3b01d24f15-kube-api-access-4z4r7\") pod \"must-gather-xxpr6\" (UID: \"9943309a-ce74-4888-a855-1e3b01d24f15\") " pod="openshift-must-gather-bpq5l/must-gather-xxpr6" Oct 30 22:39:49.101446 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:39:49.101364 5949 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bpq5l/must-gather-xxpr6" Oct 30 22:39:49.408289 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:39:49.408259 5949 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bpq5l/must-gather-xxpr6"] Oct 30 22:39:49.411611 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:39:49.411583 5949 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9943309a_ce74_4888_a855_1e3b01d24f15.slice/crio-747128564ab8c6086f8fc3ee45be2ab3e6a88496bfc45d21952db7d87c774075 WatchSource:0}: Error finding container 747128564ab8c6086f8fc3ee45be2ab3e6a88496bfc45d21952db7d87c774075: Status 404 returned error can't find the container with id 747128564ab8c6086f8fc3ee45be2ab3e6a88496bfc45d21952db7d87c774075 Oct 30 22:39:49.413523 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:39:49.413504 5949 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 22:39:50.333210 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:39:50.333167 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bpq5l/must-gather-xxpr6" event={"ID":"9943309a-ce74-4888-a855-1e3b01d24f15","Type":"ContainerStarted","Data":"747128564ab8c6086f8fc3ee45be2ab3e6a88496bfc45d21952db7d87c774075"} Oct 30 22:39:55.340114 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:39:55.340077 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:39:56.360767 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:39:56.360734 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bpq5l/must-gather-xxpr6" event={"ID":"9943309a-ce74-4888-a855-1e3b01d24f15","Type":"ContainerStarted","Data":"d992f8e766bad71b44ad360020064f9f465dd86b09d0d32992c94eace22adcee"} Oct 30 22:39:56.360767 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:39:56.360775 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bpq5l/must-gather-xxpr6" event={"ID":"9943309a-ce74-4888-a855-1e3b01d24f15","Type":"ContainerStarted","Data":"391643de1ecc09350b1772450da8c2710bda34e96917761eedfc94b62c953917"} Oct 30 22:39:56.422487 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:39:56.422414 5949 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-bpq5l/must-gather-xxpr6" podStartSLOduration=2.315538624 podStartE2EDuration="8.422396613s" totalImagesPullingTime="6.106857989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:39:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:39:56.421666431 +0000 UTC m=+1884.385475002" watchObservedRunningTime="2026-10-30 22:39:56.422396613 +0000 UTC m=+1884.386205184" Oct 30 22:40:25.335811 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:40:25.335762 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:40:28.487234 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:28.487187 5949 generic.go:356] "Generic (PLEG): container finished" podID="9943309a-ce74-4888-a855-1e3b01d24f15" containerID="391643de1ecc09350b1772450da8c2710bda34e96917761eedfc94b62c953917" exitCode=0 Oct 30 22:40:28.487658 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:28.487264 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bpq5l/must-gather-xxpr6" event={"ID":"9943309a-ce74-4888-a855-1e3b01d24f15","Type":"ContainerDied","Data":"391643de1ecc09350b1772450da8c2710bda34e96917761eedfc94b62c953917"} Oct 30 22:40:28.487658 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:28.487575 5949 scope.go:122] "RemoveContainer" containerID="391643de1ecc09350b1772450da8c2710bda34e96917761eedfc94b62c953917" Oct 30 22:40:28.619402 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:28.619367 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-bpq5l_must-gather-xxpr6_9943309a-ce74-4888-a855-1e3b01d24f15/gather/0.log" Oct 30 22:40:34.112582 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:34.112551 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-h9v2l_cd46f3e8-c752-4eed-a65d-07e060bf32b4/global-pull-secret-syncer/0.log" Oct 30 22:40:34.308537 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:34.308506 5949 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-bpq5l/must-gather-xxpr6"] Oct 30 22:40:34.308693 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:34.308650 5949 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-must-gather-bpq5l/must-gather-xxpr6" podUID="9943309a-ce74-4888-a855-1e3b01d24f15" containerName="copy" containerID="cri-o://d992f8e766bad71b44ad360020064f9f465dd86b09d0d32992c94eace22adcee" gracePeriod=2 Oct 30 22:40:34.319992 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:34.319971 5949 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-bpq5l/must-gather-xxpr6"] Oct 30 22:40:34.321393 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:40:34.321363 5949 status_manager.go:1045] "Failed to get status for pod" err="pods \"must-gather-xxpr6\" is forbidden: User \"system:node:ip-10-0-142-24.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-bpq5l\": no relationship found between node 'ip-10-0-142-24.ec2.internal' and this object" podUID="9943309a-ce74-4888-a855-1e3b01d24f15" pod="openshift-must-gather-bpq5l/must-gather-xxpr6" Oct 30 22:40:34.515765 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:34.515736 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-bpq5l_must-gather-xxpr6_9943309a-ce74-4888-a855-1e3b01d24f15/copy/0.log" Oct 30 22:40:34.516204 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:34.516172 5949 generic.go:356] "Generic (PLEG): container finished" podID="9943309a-ce74-4888-a855-1e3b01d24f15" containerID="d992f8e766bad71b44ad360020064f9f465dd86b09d0d32992c94eace22adcee" exitCode=143 Oct 30 22:40:34.544327 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:34.544307 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-bpq5l_must-gather-xxpr6_9943309a-ce74-4888-a855-1e3b01d24f15/copy/0.log" Oct 30 22:40:34.546758 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:34.546742 5949 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bpq5l/must-gather-xxpr6" Oct 30 22:40:34.559486 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:40:34.559464 5949 status_manager.go:1045] "Failed to get status for pod" err="pods \"must-gather-xxpr6\" is forbidden: User \"system:node:ip-10-0-142-24.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-bpq5l\": no relationship found between node 'ip-10-0-142-24.ec2.internal' and this object" podUID="9943309a-ce74-4888-a855-1e3b01d24f15" pod="openshift-must-gather-bpq5l/must-gather-xxpr6" Oct 30 22:40:34.652161 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:34.652137 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9943309a-ce74-4888-a855-1e3b01d24f15-must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9943309a-ce74-4888-a855-1e3b01d24f15-must-gather-output\") pod \"9943309a-ce74-4888-a855-1e3b01d24f15\" (UID: \"9943309a-ce74-4888-a855-1e3b01d24f15\") " Oct 30 22:40:34.652300 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:34.652183 5949 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9943309a-ce74-4888-a855-1e3b01d24f15-kube-api-access-4z4r7\" (UniqueName: \"kubernetes.io/projected/9943309a-ce74-4888-a855-1e3b01d24f15-kube-api-access-4z4r7\") pod \"9943309a-ce74-4888-a855-1e3b01d24f15\" (UID: \"9943309a-ce74-4888-a855-1e3b01d24f15\") " Oct 30 22:40:34.653459 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:34.653433 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9943309a-ce74-4888-a855-1e3b01d24f15-must-gather-output" pod "9943309a-ce74-4888-a855-1e3b01d24f15" (UID: "9943309a-ce74-4888-a855-1e3b01d24f15"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 22:40:34.654313 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:34.654293 5949 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9943309a-ce74-4888-a855-1e3b01d24f15-kube-api-access-4z4r7" pod "9943309a-ce74-4888-a855-1e3b01d24f15" (UID: "9943309a-ce74-4888-a855-1e3b01d24f15"). InnerVolumeSpecName "kube-api-access-4z4r7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 22:40:34.752738 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:34.752672 5949 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9943309a-ce74-4888-a855-1e3b01d24f15-must-gather-output\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:40:34.752738 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:34.752702 5949 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4z4r7\" (UniqueName: \"kubernetes.io/projected/9943309a-ce74-4888-a855-1e3b01d24f15-kube-api-access-4z4r7\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Oct 30 22:40:35.214754 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:35.214725 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-zlbzf_f6cc017b-3f50-423a-93da-e4515336a2bc/konnectivity-agent/0.log" Oct 30 22:40:35.521746 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:35.521669 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-bpq5l_must-gather-xxpr6_9943309a-ce74-4888-a855-1e3b01d24f15/copy/0.log" Oct 30 22:40:35.522066 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:35.522050 5949 scope.go:122] "RemoveContainer" containerID="d992f8e766bad71b44ad360020064f9f465dd86b09d0d32992c94eace22adcee" Oct 30 22:40:35.522145 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:35.522052 5949 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bpq5l/must-gather-xxpr6" Oct 30 22:40:35.532169 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:35.532145 5949 scope.go:122] "RemoveContainer" containerID="391643de1ecc09350b1772450da8c2710bda34e96917761eedfc94b62c953917" Oct 30 22:40:35.535133 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:40:35.535097 5949 status_manager.go:1045] "Failed to get status for pod" err="pods \"must-gather-xxpr6\" is forbidden: User \"system:node:ip-10-0-142-24.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-bpq5l\": no relationship found between node 'ip-10-0-142-24.ec2.internal' and this object" podUID="9943309a-ce74-4888-a855-1e3b01d24f15" pod="openshift-must-gather-bpq5l/must-gather-xxpr6" Oct 30 22:40:35.548134 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:40:35.548111 5949 status_manager.go:1045] "Failed to get status for pod" err="pods \"must-gather-xxpr6\" is forbidden: User \"system:node:ip-10-0-142-24.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-bpq5l\": no relationship found between node 'ip-10-0-142-24.ec2.internal' and this object" podUID="9943309a-ce74-4888-a855-1e3b01d24f15" pod="openshift-must-gather-bpq5l/must-gather-xxpr6" Oct 30 22:40:35.625702 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:35.625681 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-24.ec2.internal_7655e583c88021e605aee040501f02ec/haproxy/0.log" Oct 30 22:40:36.159297 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:36.159269 5949 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9943309a-ce74-4888-a855-1e3b01d24f15" path="/var/lib/kubelet/pods/9943309a-ce74-4888-a855-1e3b01d24f15/volumes" Oct 30 22:40:40.101621 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:40.101592 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_da6d4cc7-1aa7-4a66-8902-d7ded08379cc/alertmanager/0.log" Oct 30 22:40:40.244258 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:40.244228 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_da6d4cc7-1aa7-4a66-8902-d7ded08379cc/config-reloader/0.log" Oct 30 22:40:40.372873 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:40.372797 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_da6d4cc7-1aa7-4a66-8902-d7ded08379cc/kube-rbac-proxy-web/0.log" Oct 30 22:40:40.506808 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:40.506781 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_da6d4cc7-1aa7-4a66-8902-d7ded08379cc/kube-rbac-proxy/0.log" Oct 30 22:40:40.633440 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:40.633370 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_da6d4cc7-1aa7-4a66-8902-d7ded08379cc/kube-rbac-proxy-metric/0.log" Oct 30 22:40:40.765490 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:40.765459 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_da6d4cc7-1aa7-4a66-8902-d7ded08379cc/prom-label-proxy/0.log" Oct 30 22:40:40.895057 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:40.895036 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_da6d4cc7-1aa7-4a66-8902-d7ded08379cc/init-config-reloader/0.log" Oct 30 22:40:41.172040 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:41.171971 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-6krqs_df6b6cae-173c-461c-9195-4dce90bb6656/kube-state-metrics/0.log" Oct 30 22:40:41.300784 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:41.300755 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-6krqs_df6b6cae-173c-461c-9195-4dce90bb6656/kube-rbac-proxy-main/0.log" Oct 30 22:40:41.427398 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:41.427331 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-6krqs_df6b6cae-173c-461c-9195-4dce90bb6656/kube-rbac-proxy-self/0.log" Oct 30 22:40:41.566034 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:41.566005 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-f89f5d58d-95dk2_9ac70a73-f7f0-4b3e-bf76-09869caefea0/metrics-server/0.log" Oct 30 22:40:41.694192 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:41.694126 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-77f4f76594-29lx9_9c0d0893-e78b-48f3-bbe8-9f7d0381f42e/monitoring-plugin/0.log" Oct 30 22:40:42.621349 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:42.621321 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s7svn_68935e75-a4f0-4a54-9094-4f12b4a2dc39/node-exporter/0.log" Oct 30 22:40:42.751079 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:42.751049 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s7svn_68935e75-a4f0-4a54-9094-4f12b4a2dc39/kube-rbac-proxy/0.log" Oct 30 22:40:42.802456 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:42.802418 5949 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mprzg/perf-node-gather-daemonset-82mlj"] Oct 30 22:40:42.802809 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:42.802791 5949 cpu_manager.go:414] "containerMap: removing container" podUID="9943309a-ce74-4888-a855-1e3b01d24f15" containerName="copy" Oct 30 22:40:42.802889 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:42.802811 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9943309a-ce74-4888-a855-1e3b01d24f15" containerName="copy" Oct 30 22:40:42.802889 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:42.802837 5949 cpu_manager.go:414] "containerMap: removing container" podUID="9943309a-ce74-4888-a855-1e3b01d24f15" containerName="gather" Oct 30 22:40:42.802889 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:42.802846 5949 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9943309a-ce74-4888-a855-1e3b01d24f15" containerName="gather" Oct 30 22:40:42.803042 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:42.802943 5949 memory_manager.go:365] "RemoveStaleState removing state" podUID="9943309a-ce74-4888-a855-1e3b01d24f15" containerName="copy" Oct 30 22:40:42.803042 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:42.802958 5949 memory_manager.go:365] "RemoveStaleState removing state" podUID="9943309a-ce74-4888-a855-1e3b01d24f15" containerName="gather" Oct 30 22:40:42.808051 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:42.808029 5949 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mprzg/perf-node-gather-daemonset-82mlj" Oct 30 22:40:42.822176 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:42.822157 5949 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-mprzg\"/\"default-dockercfg-gxfc7\"" Oct 30 22:40:42.822289 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:42.822174 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mprzg\"/\"openshift-service-ca.crt\"" Oct 30 22:40:42.833843 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:42.833820 5949 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mprzg\"/\"kube-root-ca.crt\"" Oct 30 22:40:42.834558 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:42.834534 5949 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mprzg/perf-node-gather-daemonset-82mlj"] Oct 30 22:40:42.883642 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:42.883577 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s7svn_68935e75-a4f0-4a54-9094-4f12b4a2dc39/init-textfile/0.log" Oct 30 22:40:42.922907 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:42.922876 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48h8p\" (UniqueName: \"kubernetes.io/projected/d7519dfe-4425-4ad4-9372-57837dbef87d-kube-api-access-48h8p\") pod \"perf-node-gather-daemonset-82mlj\" (UID: \"d7519dfe-4425-4ad4-9372-57837dbef87d\") " pod="openshift-must-gather-mprzg/perf-node-gather-daemonset-82mlj" Oct 30 22:40:42.923051 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:42.922911 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d7519dfe-4425-4ad4-9372-57837dbef87d-podres\") pod \"perf-node-gather-daemonset-82mlj\" (UID: \"d7519dfe-4425-4ad4-9372-57837dbef87d\") " pod="openshift-must-gather-mprzg/perf-node-gather-daemonset-82mlj" Oct 30 22:40:42.923051 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:42.922953 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d7519dfe-4425-4ad4-9372-57837dbef87d-proc\") pod \"perf-node-gather-daemonset-82mlj\" (UID: \"d7519dfe-4425-4ad4-9372-57837dbef87d\") " pod="openshift-must-gather-mprzg/perf-node-gather-daemonset-82mlj" Oct 30 22:40:42.923127 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:42.923079 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d7519dfe-4425-4ad4-9372-57837dbef87d-sys\") pod \"perf-node-gather-daemonset-82mlj\" (UID: \"d7519dfe-4425-4ad4-9372-57837dbef87d\") " pod="openshift-must-gather-mprzg/perf-node-gather-daemonset-82mlj" Oct 30 22:40:42.923127 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:42.923102 5949 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d7519dfe-4425-4ad4-9372-57837dbef87d-lib-modules\") pod \"perf-node-gather-daemonset-82mlj\" (UID: \"d7519dfe-4425-4ad4-9372-57837dbef87d\") " pod="openshift-must-gather-mprzg/perf-node-gather-daemonset-82mlj" Oct 30 22:40:43.011403 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:43.011379 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-xs8x9_e129700d-0583-4316-bcee-431c57b0effa/kube-rbac-proxy-main/0.log" Oct 30 22:40:43.024267 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:43.024245 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48h8p\" (UniqueName: \"kubernetes.io/projected/d7519dfe-4425-4ad4-9372-57837dbef87d-kube-api-access-48h8p\") pod \"perf-node-gather-daemonset-82mlj\" (UID: \"d7519dfe-4425-4ad4-9372-57837dbef87d\") " pod="openshift-must-gather-mprzg/perf-node-gather-daemonset-82mlj" Oct 30 22:40:43.024381 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:43.024281 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d7519dfe-4425-4ad4-9372-57837dbef87d-podres\") pod \"perf-node-gather-daemonset-82mlj\" (UID: \"d7519dfe-4425-4ad4-9372-57837dbef87d\") " pod="openshift-must-gather-mprzg/perf-node-gather-daemonset-82mlj" Oct 30 22:40:43.024381 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:43.024300 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d7519dfe-4425-4ad4-9372-57837dbef87d-proc\") pod \"perf-node-gather-daemonset-82mlj\" (UID: \"d7519dfe-4425-4ad4-9372-57837dbef87d\") " pod="openshift-must-gather-mprzg/perf-node-gather-daemonset-82mlj" Oct 30 22:40:43.024381 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:43.024351 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d7519dfe-4425-4ad4-9372-57837dbef87d-sys\") pod \"perf-node-gather-daemonset-82mlj\" (UID: \"d7519dfe-4425-4ad4-9372-57837dbef87d\") " pod="openshift-must-gather-mprzg/perf-node-gather-daemonset-82mlj" Oct 30 22:40:43.024381 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:43.024369 5949 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d7519dfe-4425-4ad4-9372-57837dbef87d-lib-modules\") pod \"perf-node-gather-daemonset-82mlj\" (UID: \"d7519dfe-4425-4ad4-9372-57837dbef87d\") " pod="openshift-must-gather-mprzg/perf-node-gather-daemonset-82mlj" Oct 30 22:40:43.024521 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:43.024384 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d7519dfe-4425-4ad4-9372-57837dbef87d-podres\") pod \"perf-node-gather-daemonset-82mlj\" (UID: \"d7519dfe-4425-4ad4-9372-57837dbef87d\") " pod="openshift-must-gather-mprzg/perf-node-gather-daemonset-82mlj" Oct 30 22:40:43.024521 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:43.024444 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d7519dfe-4425-4ad4-9372-57837dbef87d-sys\") pod \"perf-node-gather-daemonset-82mlj\" (UID: \"d7519dfe-4425-4ad4-9372-57837dbef87d\") " pod="openshift-must-gather-mprzg/perf-node-gather-daemonset-82mlj" Oct 30 22:40:43.024521 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:43.024454 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d7519dfe-4425-4ad4-9372-57837dbef87d-lib-modules\") pod \"perf-node-gather-daemonset-82mlj\" (UID: \"d7519dfe-4425-4ad4-9372-57837dbef87d\") " pod="openshift-must-gather-mprzg/perf-node-gather-daemonset-82mlj" Oct 30 22:40:43.024521 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:43.024444 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d7519dfe-4425-4ad4-9372-57837dbef87d-proc\") pod \"perf-node-gather-daemonset-82mlj\" (UID: \"d7519dfe-4425-4ad4-9372-57837dbef87d\") " pod="openshift-must-gather-mprzg/perf-node-gather-daemonset-82mlj" Oct 30 22:40:43.047628 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:43.047605 5949 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-48h8p\" (UniqueName: \"kubernetes.io/projected/d7519dfe-4425-4ad4-9372-57837dbef87d-kube-api-access-48h8p\") pod \"perf-node-gather-daemonset-82mlj\" (UID: \"d7519dfe-4425-4ad4-9372-57837dbef87d\") " pod="openshift-must-gather-mprzg/perf-node-gather-daemonset-82mlj" Oct 30 22:40:43.120272 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:43.120248 5949 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mprzg/perf-node-gather-daemonset-82mlj" Oct 30 22:40:43.136483 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:43.136422 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-xs8x9_e129700d-0583-4316-bcee-431c57b0effa/kube-rbac-proxy-self/0.log" Oct 30 22:40:43.263099 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:43.263066 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-xs8x9_e129700d-0583-4316-bcee-431c57b0effa/openshift-state-metrics/0.log" Oct 30 22:40:43.397665 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:43.397636 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e80ba5e9-a0f2-4303-91b2-41fe096f88fe/prometheus/0.log" Oct 30 22:40:43.413681 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:43.413656 5949 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mprzg/perf-node-gather-daemonset-82mlj"] Oct 30 22:40:43.417592 ip-10-0-142-24 kubenswrapper[5949]: W1030 22:40:43.417563 5949 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd7519dfe_4425_4ad4_9372_57837dbef87d.slice/crio-2434096155a092eb2ada42632b5d27b8e00385410c62cdd299c8cc49a7fa5a19 WatchSource:0}: Error finding container 2434096155a092eb2ada42632b5d27b8e00385410c62cdd299c8cc49a7fa5a19: Status 404 returned error can't find the container with id 2434096155a092eb2ada42632b5d27b8e00385410c62cdd299c8cc49a7fa5a19 Oct 30 22:40:43.524426 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:43.524406 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e80ba5e9-a0f2-4303-91b2-41fe096f88fe/config-reloader/0.log" Oct 30 22:40:43.551155 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:43.551131 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mprzg/perf-node-gather-daemonset-82mlj" event={"ID":"d7519dfe-4425-4ad4-9372-57837dbef87d","Type":"ContainerStarted","Data":"1ac79dc12c7147e45d78ce1ddf4ff6f41b0f575cdc30920029b4e095ca896b06"} Oct 30 22:40:43.551258 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:43.551174 5949 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mprzg/perf-node-gather-daemonset-82mlj" event={"ID":"d7519dfe-4425-4ad4-9372-57837dbef87d","Type":"ContainerStarted","Data":"2434096155a092eb2ada42632b5d27b8e00385410c62cdd299c8cc49a7fa5a19"} Oct 30 22:40:43.551321 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:43.551302 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-mprzg/perf-node-gather-daemonset-82mlj" Oct 30 22:40:43.591486 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:43.591444 5949 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-mprzg/perf-node-gather-daemonset-82mlj" podStartSLOduration=1.591432557 podStartE2EDuration="1.591432557s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:40:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:40:43.59100394 +0000 UTC m=+1931.554812509" watchObservedRunningTime="2026-10-30 22:40:43.591432557 +0000 UTC m=+1931.555241124" Oct 30 22:40:43.650037 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:43.649955 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e80ba5e9-a0f2-4303-91b2-41fe096f88fe/thanos-sidecar/0.log" Oct 30 22:40:43.777995 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:43.777973 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e80ba5e9-a0f2-4303-91b2-41fe096f88fe/kube-rbac-proxy-web/0.log" Oct 30 22:40:43.906859 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:43.906800 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e80ba5e9-a0f2-4303-91b2-41fe096f88fe/kube-rbac-proxy/0.log" Oct 30 22:40:44.032928 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:44.032903 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e80ba5e9-a0f2-4303-91b2-41fe096f88fe/kube-rbac-proxy-thanos/0.log" Oct 30 22:40:44.159045 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:44.158984 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e80ba5e9-a0f2-4303-91b2-41fe096f88fe/init-config-reloader/0.log" Oct 30 22:40:44.678330 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:44.678303 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-74754b6cd9-49r6n_fb3d3e48-51f2-4503-8026-79787ee98754/telemeter-client/0.log" Oct 30 22:40:44.802420 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:44.802392 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-74754b6cd9-49r6n_fb3d3e48-51f2-4503-8026-79787ee98754/reload/0.log" Oct 30 22:40:44.927832 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:44.927805 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-74754b6cd9-49r6n_fb3d3e48-51f2-4503-8026-79787ee98754/kube-rbac-proxy/0.log" Oct 30 22:40:45.081425 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:45.081351 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5ddf98bd44-xfj64_1b444016-092c-472b-979a-4a7b16271e27/thanos-query/0.log" Oct 30 22:40:45.207170 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:45.207143 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5ddf98bd44-xfj64_1b444016-092c-472b-979a-4a7b16271e27/kube-rbac-proxy-web/0.log" Oct 30 22:40:45.340038 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:45.339968 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5ddf98bd44-xfj64_1b444016-092c-472b-979a-4a7b16271e27/kube-rbac-proxy/0.log" Oct 30 22:40:45.467919 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:45.467890 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5ddf98bd44-xfj64_1b444016-092c-472b-979a-4a7b16271e27/prom-label-proxy/0.log" Oct 30 22:40:45.593531 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:45.593463 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5ddf98bd44-xfj64_1b444016-092c-472b-979a-4a7b16271e27/kube-rbac-proxy-rules/0.log" Oct 30 22:40:45.722897 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:45.722873 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5ddf98bd44-xfj64_1b444016-092c-472b-979a-4a7b16271e27/kube-rbac-proxy-metrics/0.log" Oct 30 22:40:49.564046 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:49.564011 5949 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-mprzg/perf-node-gather-daemonset-82mlj" Oct 30 22:40:52.493123 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:52.493097 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pl4x2_c373c767-d743-4819-be7c-4c9c830eb9db/dns/0.log" Oct 30 22:40:52.620359 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:52.620332 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pl4x2_c373c767-d743-4819-be7c-4c9c830eb9db/kube-rbac-proxy/0.log" Oct 30 22:40:53.308186 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:53.308161 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pnr6k_6b042dce-1fff-41a2-83dd-ae18f3098207/dns-node-resolver/0.log" Oct 30 22:40:54.521988 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:54.521963 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dbtlp_11464051-0aa4-4324-b2c7-7ea2096bbed6/node-ca/0.log" Oct 30 22:40:55.336262 ip-10-0-142-24 kubenswrapper[5949]: E1030 22:40:55.336227 5949 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-pn4kg" containerName="extractor" Oct 30 22:40:57.441407 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:57.441381 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9mkk7_f64980b7-13c0-40c3-aa3f-277916f24518/serve-healthcheck-canary/0.log" Oct 30 22:40:59.493117 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:59.493094 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pn4kg_5dfaf077-3637-4cc9-8681-466c4e2cc5f4/kube-rbac-proxy/0.log" Oct 30 22:40:59.619393 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:59.619367 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pn4kg_5dfaf077-3637-4cc9-8681-466c4e2cc5f4/exporter/0.log" Oct 30 22:40:59.743935 ip-10-0-142-24 kubenswrapper[5949]: I1030 22:40:59.743861 5949 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pn4kg_5dfaf077-3637-4cc9-8681-466c4e2cc5f4/extractor/0.log"