Nov 03 21:44:41.885444 ip-10-0-137-7 systemd[1]: Starting Kubernetes Kubelet... Nov 03 21:44:42.264348 ip-10-0-137-7 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 03 21:44:42.264348 ip-10-0-137-7 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Nov 03 21:44:42.264348 ip-10-0-137-7 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 03 21:44:42.264348 ip-10-0-137-7 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 03 21:44:42.269069 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.268968 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 03 21:44:42.269069 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269068 2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 03 21:44:42.269141 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269073 2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 03 21:44:42.269141 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269076 2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 03 21:44:42.269141 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269079 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 03 21:44:42.269141 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269082 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 03 21:44:42.269141 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269085 2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 03 21:44:42.269141 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269088 2578 feature_gate.go:386] unrecognized feature gate: EVPN Nov 03 21:44:42.269141 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269090 2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 03 21:44:42.269141 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269093 2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 03 21:44:42.269141 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269096 2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 03 21:44:42.269141 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269098 2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 03 21:44:42.269141 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269101 2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 03 21:44:42.269141 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269104 2578 feature_gate.go:386] unrecognized feature gate: Example2 Nov 03 21:44:42.269141 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269107 2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 03 21:44:42.269141 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269110 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 03 21:44:42.269141 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269113 2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 03 21:44:42.269141 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269116 2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 03 21:44:42.269141 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269118 2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 03 21:44:42.269141 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269121 2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 03 21:44:42.269141 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269123 2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 03 21:44:42.269141 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269126 2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 03 21:44:42.269604 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269128 2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 03 21:44:42.269604 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269132 2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 03 21:44:42.269604 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269134 2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 03 21:44:42.269604 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269137 2578 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 03 21:44:42.269604 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269139 2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 03 21:44:42.269604 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269142 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 03 21:44:42.269604 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269145 2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 03 21:44:42.269604 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269148 2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 03 21:44:42.269604 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269151 2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 03 21:44:42.269604 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269153 2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 03 21:44:42.269604 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269156 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 03 21:44:42.269604 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269159 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 03 21:44:42.269604 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269162 2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 03 21:44:42.269604 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269165 2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 03 21:44:42.269604 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269167 2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 03 21:44:42.269604 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269170 2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 03 21:44:42.269604 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269172 2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 03 21:44:42.269604 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269176 2578 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 03 21:44:42.269604 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269179 2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 03 21:44:42.270069 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269182 2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 03 21:44:42.270069 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269184 2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 03 21:44:42.270069 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269187 2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 03 21:44:42.270069 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269189 2578 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 03 21:44:42.270069 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269192 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 03 21:44:42.270069 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269194 2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 03 21:44:42.270069 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269197 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 03 21:44:42.270069 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269200 2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 03 21:44:42.270069 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269202 2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 03 21:44:42.270069 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269205 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 03 21:44:42.270069 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269207 2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 03 21:44:42.270069 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269209 2578 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 03 21:44:42.270069 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269212 2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 03 21:44:42.270069 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269214 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 03 21:44:42.270069 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269218 2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 03 21:44:42.270069 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269220 2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 03 21:44:42.270069 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269223 2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 03 21:44:42.270069 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269226 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 03 21:44:42.270069 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269228 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 03 21:44:42.270520 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269231 2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 03 21:44:42.270520 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269233 2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 03 21:44:42.270520 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269236 2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 03 21:44:42.270520 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269238 2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 03 21:44:42.270520 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269241 2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 03 21:44:42.270520 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269244 2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 03 21:44:42.270520 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269247 2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 03 21:44:42.270520 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269249 2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 03 21:44:42.270520 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269251 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 03 21:44:42.270520 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269254 2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 03 21:44:42.270520 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269257 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 03 21:44:42.270520 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269259 2578 feature_gate.go:386] unrecognized feature gate: Example Nov 03 21:44:42.270520 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269263 2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 03 21:44:42.270520 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269266 2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 03 21:44:42.270520 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269269 2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 03 21:44:42.270520 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269271 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 03 21:44:42.270520 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269274 2578 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 03 21:44:42.270520 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269276 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 03 21:44:42.270520 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269279 2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 03 21:44:42.270520 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269281 2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 03 21:44:42.271012 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269284 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 03 21:44:42.271012 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269286 2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 03 21:44:42.271012 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269289 2578 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 03 21:44:42.271012 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269292 2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 03 21:44:42.271012 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269294 2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 03 21:44:42.271012 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269297 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 03 21:44:42.271012 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269299 2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 03 21:44:42.271012 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269304 2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 03 21:44:42.271012 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269306 2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 03 21:44:42.271012 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269310 2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 03 21:44:42.271012 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269313 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 03 21:44:42.271012 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269316 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 03 21:44:42.271012 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269319 2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 03 21:44:42.271012 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269321 2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 03 21:44:42.271012 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269326 2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 03 21:44:42.271012 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269330 2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 03 21:44:42.271012 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269333 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 03 21:44:42.271012 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269336 2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 03 21:44:42.271012 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269339 2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 03 21:44:42.271456 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269341 2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 03 21:44:42.271456 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269344 2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 03 21:44:42.271456 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269347 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 03 21:44:42.271456 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269349 2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 03 21:44:42.271456 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269352 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 03 21:44:42.271456 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269357 2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 03 21:44:42.271456 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.269361 2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 03 21:44:42.271456 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270198 2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 03 21:44:42.271456 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270206 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 03 21:44:42.271456 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270211 2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 03 21:44:42.271456 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270215 2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 03 21:44:42.271456 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270218 2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 03 21:44:42.271456 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270221 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 03 21:44:42.271456 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270225 2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 03 21:44:42.271456 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270227 2578 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 03 21:44:42.271456 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270230 2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 03 21:44:42.271456 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270233 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 03 21:44:42.271456 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270236 2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 03 21:44:42.271456 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270238 2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 03 21:44:42.271958 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270241 2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 03 21:44:42.271958 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270244 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 03 21:44:42.271958 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270247 2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 03 21:44:42.271958 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270250 2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 03 21:44:42.271958 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270253 2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 03 21:44:42.271958 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270256 2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 03 21:44:42.271958 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270258 2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 03 21:44:42.271958 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270261 2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 03 21:44:42.271958 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270264 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 03 21:44:42.271958 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270266 2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 03 21:44:42.271958 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270269 2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 03 21:44:42.271958 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270272 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 03 21:44:42.271958 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270274 2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 03 21:44:42.271958 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270277 2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 03 21:44:42.271958 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270279 2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 03 21:44:42.271958 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270282 2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 03 21:44:42.271958 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270284 2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 03 21:44:42.271958 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270287 2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 03 21:44:42.271958 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270290 2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 03 21:44:42.272417 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270292 2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 03 21:44:42.272417 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270295 2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 03 21:44:42.272417 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270298 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 03 21:44:42.272417 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270301 2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 03 21:44:42.272417 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270303 2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 03 21:44:42.272417 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270306 2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 03 21:44:42.272417 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270309 2578 feature_gate.go:386] unrecognized feature gate: EVPN Nov 03 21:44:42.272417 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270311 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 03 21:44:42.272417 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270314 2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 03 21:44:42.272417 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270316 2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 03 21:44:42.272417 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270319 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 03 21:44:42.272417 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270321 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 03 21:44:42.272417 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270324 2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 03 21:44:42.272417 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270326 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 03 21:44:42.272417 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270329 2578 feature_gate.go:386] unrecognized feature gate: Example2 Nov 03 21:44:42.272417 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270332 2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 03 21:44:42.272417 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270335 2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 03 21:44:42.272417 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270337 2578 feature_gate.go:386] unrecognized feature gate: Example Nov 03 21:44:42.272417 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270340 2578 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 03 21:44:42.272417 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270342 2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 03 21:44:42.272897 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270345 2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 03 21:44:42.272897 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270347 2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 03 21:44:42.272897 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270349 2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 03 21:44:42.272897 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270352 2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 03 21:44:42.272897 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270354 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 03 21:44:42.272897 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270357 2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 03 21:44:42.272897 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270360 2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 03 21:44:42.272897 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270362 2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 03 21:44:42.272897 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270365 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 03 21:44:42.272897 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270367 2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 03 21:44:42.272897 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270369 2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 03 21:44:42.272897 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270372 2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 03 21:44:42.272897 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270374 2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 03 21:44:42.272897 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270378 2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 03 21:44:42.272897 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270381 2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 03 21:44:42.272897 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270383 2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 03 21:44:42.272897 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270388 2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 03 21:44:42.272897 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270392 2578 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 03 21:44:42.272897 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270396 2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 03 21:44:42.273378 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270399 2578 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 03 21:44:42.273378 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270403 2578 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 03 21:44:42.273378 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270406 2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 03 21:44:42.273378 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270408 2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 03 21:44:42.273378 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270411 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 03 21:44:42.273378 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270413 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 03 21:44:42.273378 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270416 2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 03 21:44:42.273378 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270418 2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 03 21:44:42.273378 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270421 2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 03 21:44:42.273378 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270424 2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 03 21:44:42.273378 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270427 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 03 21:44:42.273378 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270429 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 03 21:44:42.273378 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270432 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 03 21:44:42.273378 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270434 2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 03 21:44:42.273378 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270437 2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 03 21:44:42.273378 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270439 2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 03 21:44:42.273378 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270442 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 03 21:44:42.273378 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270444 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 03 21:44:42.273378 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270447 2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 03 21:44:42.273378 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270449 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 03 21:44:42.273849 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270452 2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 03 21:44:42.273849 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270454 2578 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 03 21:44:42.273849 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270456 2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 03 21:44:42.273849 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270459 2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 03 21:44:42.273849 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270461 2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 03 21:44:42.273849 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270464 2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 03 21:44:42.273849 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270466 2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 03 21:44:42.273849 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270469 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 03 21:44:42.273849 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270472 2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 03 21:44:42.273849 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270475 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 03 21:44:42.273849 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270478 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 03 21:44:42.273849 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270481 2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 03 21:44:42.273849 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270484 2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 03 21:44:42.273849 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270487 2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 03 21:44:42.273849 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270490 2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 03 21:44:42.273849 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270493 2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 03 21:44:42.273849 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270970 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 03 21:44:42.273849 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270975 2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 03 21:44:42.273849 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270978 2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 03 21:44:42.273849 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270995 2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 03 21:44:42.274337 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.270998 2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 03 21:44:42.274337 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271001 2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 03 21:44:42.274337 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271004 2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 03 21:44:42.274337 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271006 2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 03 21:44:42.274337 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271009 2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 03 21:44:42.274337 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271012 2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 03 21:44:42.274337 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271015 2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 03 21:44:42.274337 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271017 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 03 21:44:42.274337 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271020 2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 03 21:44:42.274337 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271023 2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 03 21:44:42.274337 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271026 2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 03 21:44:42.274337 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271029 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 03 21:44:42.274337 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271031 2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 03 21:44:42.274337 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271034 2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 03 21:44:42.274337 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271036 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 03 21:44:42.274337 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271039 2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 03 21:44:42.274337 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271041 2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 03 21:44:42.274337 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271044 2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 03 21:44:42.274337 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271046 2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 03 21:44:42.274337 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271049 2578 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 03 21:44:42.274803 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271053 2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 03 21:44:42.274803 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271058 2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 03 21:44:42.274803 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271060 2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 03 21:44:42.274803 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271063 2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 03 21:44:42.274803 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271066 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 03 21:44:42.274803 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271068 2578 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 03 21:44:42.274803 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271071 2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 03 21:44:42.274803 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271073 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 03 21:44:42.274803 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271076 2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 03 21:44:42.274803 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271078 2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 03 21:44:42.274803 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271081 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 03 21:44:42.274803 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271083 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 03 21:44:42.274803 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271086 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 03 21:44:42.274803 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271088 2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 03 21:44:42.274803 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271091 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 03 21:44:42.274803 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271093 2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 03 21:44:42.274803 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271096 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 03 21:44:42.274803 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271098 2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 03 21:44:42.274803 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271101 2578 feature_gate.go:386] unrecognized feature gate: Example Nov 03 21:44:42.274803 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271103 2578 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 03 21:44:42.275294 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271106 2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 03 21:44:42.275294 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271109 2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 03 21:44:42.275294 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271111 2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 03 21:44:42.275294 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271114 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 03 21:44:42.275294 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271116 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 03 21:44:42.275294 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271119 2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 03 21:44:42.275294 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271122 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 03 21:44:42.275294 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271124 2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 03 21:44:42.275294 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271127 2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 03 21:44:42.275294 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271130 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 03 21:44:42.275294 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271132 2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 03 21:44:42.275294 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271135 2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 03 21:44:42.275294 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271138 2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 03 21:44:42.275294 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271142 2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 03 21:44:42.275294 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271145 2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 03 21:44:42.275294 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271147 2578 feature_gate.go:386] unrecognized feature gate: Example2 Nov 03 21:44:42.275294 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271151 2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 03 21:44:42.275294 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271156 2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 03 21:44:42.275294 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271158 2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 03 21:44:42.275750 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271161 2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 03 21:44:42.275750 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271164 2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 03 21:44:42.275750 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271166 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 03 21:44:42.275750 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271169 2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 03 21:44:42.275750 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271171 2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 03 21:44:42.275750 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271174 2578 feature_gate.go:386] unrecognized feature gate: EVPN Nov 03 21:44:42.275750 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271177 2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 03 21:44:42.275750 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271179 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 03 21:44:42.275750 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271182 2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 03 21:44:42.275750 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271184 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 03 21:44:42.275750 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271187 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 03 21:44:42.275750 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271189 2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 03 21:44:42.275750 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271192 2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 03 21:44:42.275750 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271194 2578 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 03 21:44:42.275750 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271197 2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 03 21:44:42.275750 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271199 2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 03 21:44:42.275750 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271202 2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 03 21:44:42.275750 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271205 2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 03 21:44:42.275750 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271207 2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 03 21:44:42.276268 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271210 2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 03 21:44:42.276268 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271212 2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 03 21:44:42.276268 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271214 2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 03 21:44:42.276268 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271217 2578 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 03 21:44:42.276268 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271219 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 03 21:44:42.276268 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271222 2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 03 21:44:42.276268 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271224 2578 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 03 21:44:42.276268 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271229 2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 03 21:44:42.276268 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271232 2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 03 21:44:42.276268 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271235 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 03 21:44:42.276268 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271238 2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 03 21:44:42.276268 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271241 2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 03 21:44:42.276268 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271243 2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 03 21:44:42.276268 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271246 2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 03 21:44:42.276268 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271248 2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 03 21:44:42.276268 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271251 2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 03 21:44:42.276268 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271253 2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 03 21:44:42.276268 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271256 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 03 21:44:42.276268 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271258 2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 03 21:44:42.276725 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271263 2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 03 21:44:42.276725 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271267 2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 03 21:44:42.276725 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271270 2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 03 21:44:42.276725 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271273 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 03 21:44:42.276725 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.271275 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 03 21:44:42.276725 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271362 2578 flags.go:64] FLAG: --address="0.0.0.0" Nov 03 21:44:42.276725 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271369 2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Nov 03 21:44:42.276725 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271375 2578 flags.go:64] FLAG: --anonymous-auth="true" Nov 03 21:44:42.276725 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271380 2578 flags.go:64] FLAG: --application-metrics-count-limit="100" Nov 03 21:44:42.276725 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271385 2578 flags.go:64] FLAG: --authentication-token-webhook="false" Nov 03 21:44:42.276725 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271388 2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Nov 03 21:44:42.276725 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271392 2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Nov 03 21:44:42.276725 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271397 2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Nov 03 21:44:42.276725 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271400 2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Nov 03 21:44:42.276725 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271403 2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Nov 03 21:44:42.276725 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271406 2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Nov 03 21:44:42.276725 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271410 2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Nov 03 21:44:42.276725 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271413 2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Nov 03 21:44:42.276725 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271416 2578 flags.go:64] FLAG: --cgroup-root="" Nov 03 21:44:42.276725 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271419 2578 flags.go:64] FLAG: --cgroups-per-qos="true" Nov 03 21:44:42.276725 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271424 2578 flags.go:64] FLAG: --client-ca-file="" Nov 03 21:44:42.276725 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271426 2578 flags.go:64] FLAG: --cloud-provider="external" Nov 03 21:44:42.276725 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271432 2578 flags.go:64] FLAG: --cluster-dns="[]" Nov 03 21:44:42.277290 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271437 2578 flags.go:64] FLAG: --cluster-domain="" Nov 03 21:44:42.277290 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271440 2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Nov 03 21:44:42.277290 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271443 2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Nov 03 21:44:42.277290 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271446 2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Nov 03 21:44:42.277290 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271450 2578 flags.go:64] FLAG: --container-log-max-files="5" Nov 03 21:44:42.277290 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271454 2578 flags.go:64] FLAG: --container-log-max-size="10Mi" Nov 03 21:44:42.277290 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271457 2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Nov 03 21:44:42.277290 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271460 2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Nov 03 21:44:42.277290 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271464 2578 flags.go:64] FLAG: --containerd-namespace="k8s.io" Nov 03 21:44:42.277290 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271467 2578 flags.go:64] FLAG: --contention-profiling="false" Nov 03 21:44:42.277290 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271471 2578 flags.go:64] FLAG: --cpu-cfs-quota="true" Nov 03 21:44:42.277290 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271474 2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Nov 03 21:44:42.277290 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271477 2578 flags.go:64] FLAG: --cpu-manager-policy="none" Nov 03 21:44:42.277290 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271480 2578 flags.go:64] FLAG: --cpu-manager-policy-options="" Nov 03 21:44:42.277290 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271485 2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Nov 03 21:44:42.277290 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271488 2578 flags.go:64] FLAG: --enable-controller-attach-detach="true" Nov 03 21:44:42.277290 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271492 2578 flags.go:64] FLAG: --enable-debugging-handlers="true" Nov 03 21:44:42.277290 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271495 2578 flags.go:64] FLAG: --enable-load-reader="false" Nov 03 21:44:42.277290 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271498 2578 flags.go:64] FLAG: --enable-server="true" Nov 03 21:44:42.277290 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271501 2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Nov 03 21:44:42.277290 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271506 2578 flags.go:64] FLAG: --event-burst="100" Nov 03 21:44:42.277290 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271509 2578 flags.go:64] FLAG: --event-qps="50" Nov 03 21:44:42.277290 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271512 2578 flags.go:64] FLAG: --event-storage-age-limit="default=0" Nov 03 21:44:42.277290 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271515 2578 flags.go:64] FLAG: --event-storage-event-limit="default=0" Nov 03 21:44:42.277892 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271519 2578 flags.go:64] FLAG: --eviction-hard="" Nov 03 21:44:42.277892 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271523 2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Nov 03 21:44:42.277892 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271526 2578 flags.go:64] FLAG: --eviction-minimum-reclaim="" Nov 03 21:44:42.277892 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271529 2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Nov 03 21:44:42.277892 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271532 2578 flags.go:64] FLAG: --eviction-soft="" Nov 03 21:44:42.277892 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271537 2578 flags.go:64] FLAG: --eviction-soft-grace-period="" Nov 03 21:44:42.277892 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271540 2578 flags.go:64] FLAG: --exit-on-lock-contention="false" Nov 03 21:44:42.277892 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271545 2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Nov 03 21:44:42.277892 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271548 2578 flags.go:64] FLAG: --experimental-mounter-path="" Nov 03 21:44:42.277892 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271551 2578 flags.go:64] FLAG: --fail-cgroupv1="true" Nov 03 21:44:42.277892 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271554 2578 flags.go:64] FLAG: --fail-swap-on="true" Nov 03 21:44:42.277892 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271557 2578 flags.go:64] FLAG: --feature-gates="" Nov 03 21:44:42.277892 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271561 2578 flags.go:64] FLAG: --file-check-frequency="20s" Nov 03 21:44:42.277892 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271564 2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Nov 03 21:44:42.277892 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271568 2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Nov 03 21:44:42.277892 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271571 2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Nov 03 21:44:42.277892 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271574 2578 flags.go:64] FLAG: --healthz-port="10248" Nov 03 21:44:42.277892 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271577 2578 flags.go:64] FLAG: --help="false" Nov 03 21:44:42.277892 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271580 2578 flags.go:64] FLAG: --hostname-override="ip-10-0-137-7.us-east-2.compute.internal" Nov 03 21:44:42.277892 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271584 2578 flags.go:64] FLAG: --housekeeping-interval="10s" Nov 03 21:44:42.277892 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271587 2578 flags.go:64] FLAG: --http-check-frequency="20s" Nov 03 21:44:42.277892 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271590 2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Nov 03 21:44:42.277892 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271593 2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Nov 03 21:44:42.278448 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271597 2578 flags.go:64] FLAG: --image-gc-high-threshold="85" Nov 03 21:44:42.278448 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271600 2578 flags.go:64] FLAG: --image-gc-low-threshold="80" Nov 03 21:44:42.278448 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271603 2578 flags.go:64] FLAG: --image-service-endpoint="" Nov 03 21:44:42.278448 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271605 2578 flags.go:64] FLAG: --kernel-memcg-notification="false" Nov 03 21:44:42.278448 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271608 2578 flags.go:64] FLAG: --kube-api-burst="100" Nov 03 21:44:42.278448 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271611 2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Nov 03 21:44:42.278448 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271614 2578 flags.go:64] FLAG: --kube-api-qps="50" Nov 03 21:44:42.278448 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271617 2578 flags.go:64] FLAG: --kube-reserved="" Nov 03 21:44:42.278448 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271620 2578 flags.go:64] FLAG: --kube-reserved-cgroup="" Nov 03 21:44:42.278448 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271623 2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Nov 03 21:44:42.278448 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271626 2578 flags.go:64] FLAG: --kubelet-cgroups="" Nov 03 21:44:42.278448 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271629 2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Nov 03 21:44:42.278448 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271632 2578 flags.go:64] FLAG: --lock-file="" Nov 03 21:44:42.278448 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271635 2578 flags.go:64] FLAG: --log-cadvisor-usage="false" Nov 03 21:44:42.278448 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271639 2578 flags.go:64] FLAG: --log-flush-frequency="5s" Nov 03 21:44:42.278448 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271642 2578 flags.go:64] FLAG: --log-json-info-buffer-size="0" Nov 03 21:44:42.278448 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271649 2578 flags.go:64] FLAG: --log-json-split-stream="false" Nov 03 21:44:42.278448 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271652 2578 flags.go:64] FLAG: --log-text-info-buffer-size="0" Nov 03 21:44:42.278448 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271656 2578 flags.go:64] FLAG: --log-text-split-stream="false" Nov 03 21:44:42.278448 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271659 2578 flags.go:64] FLAG: --logging-format="text" Nov 03 21:44:42.278448 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271662 2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Nov 03 21:44:42.278448 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271666 2578 flags.go:64] FLAG: --make-iptables-util-chains="true" Nov 03 21:44:42.278448 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271669 2578 flags.go:64] FLAG: --manifest-url="" Nov 03 21:44:42.278448 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271672 2578 flags.go:64] FLAG: --manifest-url-header="" Nov 03 21:44:42.278448 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271676 2578 flags.go:64] FLAG: --max-housekeeping-interval="15s" Nov 03 21:44:42.279067 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271679 2578 flags.go:64] FLAG: --max-open-files="1000000" Nov 03 21:44:42.279067 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271683 2578 flags.go:64] FLAG: --max-pods="110" Nov 03 21:44:42.279067 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271686 2578 flags.go:64] FLAG: --maximum-dead-containers="-1" Nov 03 21:44:42.279067 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271689 2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Nov 03 21:44:42.279067 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271692 2578 flags.go:64] FLAG: --memory-manager-policy="None" Nov 03 21:44:42.279067 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271696 2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Nov 03 21:44:42.279067 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271699 2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Nov 03 21:44:42.279067 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271702 2578 flags.go:64] FLAG: --node-ip="0.0.0.0" Nov 03 21:44:42.279067 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271705 2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Nov 03 21:44:42.279067 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271711 2578 flags.go:64] FLAG: --node-status-max-images="50" Nov 03 21:44:42.279067 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271715 2578 flags.go:64] FLAG: --node-status-update-frequency="10s" Nov 03 21:44:42.279067 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271718 2578 flags.go:64] FLAG: --oom-score-adj="-999" Nov 03 21:44:42.279067 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271722 2578 flags.go:64] FLAG: --pod-cidr="" Nov 03 21:44:42.279067 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271724 2578 flags.go:64] FLAG: --pod-manifest-path="" Nov 03 21:44:42.279067 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271727 2578 flags.go:64] FLAG: --pod-max-pids="-1" Nov 03 21:44:42.279067 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271730 2578 flags.go:64] FLAG: --pods-per-core="0" Nov 03 21:44:42.279067 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271733 2578 flags.go:64] FLAG: --port="10250" Nov 03 21:44:42.279067 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271736 2578 flags.go:64] FLAG: --protect-kernel-defaults="false" Nov 03 21:44:42.279067 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271739 2578 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-0aae6af7f38c14baa" Nov 03 21:44:42.279067 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271743 2578 flags.go:64] FLAG: --qos-reserved="" Nov 03 21:44:42.279067 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271746 2578 flags.go:64] FLAG: --read-only-port="10255" Nov 03 21:44:42.279067 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271750 2578 flags.go:64] FLAG: --register-node="true" Nov 03 21:44:42.279067 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271753 2578 flags.go:64] FLAG: --register-with-taints="" Nov 03 21:44:42.279067 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271758 2578 flags.go:64] FLAG: --registry-burst="10" Nov 03 21:44:42.279067 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271761 2578 flags.go:64] FLAG: --registry-qps="5" Nov 03 21:44:42.279067 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271764 2578 flags.go:64] FLAG: --reserved-cpus="" Nov 03 21:44:42.279733 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271768 2578 flags.go:64] FLAG: --reserved-memory="" Nov 03 21:44:42.279733 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271772 2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Nov 03 21:44:42.279733 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271775 2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Nov 03 21:44:42.279733 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271778 2578 flags.go:64] FLAG: --rotate-certificates="false" Nov 03 21:44:42.279733 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271781 2578 flags.go:64] FLAG: --rotate-server-certificates="false" Nov 03 21:44:42.279733 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271783 2578 flags.go:64] FLAG: --runonce="false" Nov 03 21:44:42.279733 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271786 2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Nov 03 21:44:42.279733 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271790 2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Nov 03 21:44:42.279733 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271793 2578 flags.go:64] FLAG: --seccomp-default="false" Nov 03 21:44:42.279733 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271796 2578 flags.go:64] FLAG: --serialize-image-pulls="true" Nov 03 21:44:42.279733 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271799 2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Nov 03 21:44:42.279733 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271802 2578 flags.go:64] FLAG: --storage-driver-db="cadvisor" Nov 03 21:44:42.279733 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271805 2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Nov 03 21:44:42.279733 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271808 2578 flags.go:64] FLAG: --storage-driver-password="root" Nov 03 21:44:42.279733 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271812 2578 flags.go:64] FLAG: --storage-driver-secure="false" Nov 03 21:44:42.279733 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271815 2578 flags.go:64] FLAG: --storage-driver-table="stats" Nov 03 21:44:42.279733 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271817 2578 flags.go:64] FLAG: --storage-driver-user="root" Nov 03 21:44:42.279733 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271821 2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Nov 03 21:44:42.279733 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271824 2578 flags.go:64] FLAG: --sync-frequency="1m0s" Nov 03 21:44:42.279733 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271826 2578 flags.go:64] FLAG: --system-cgroups="" Nov 03 21:44:42.279733 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271829 2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Nov 03 21:44:42.279733 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271834 2578 flags.go:64] FLAG: --system-reserved-cgroup="" Nov 03 21:44:42.279733 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271837 2578 flags.go:64] FLAG: --tls-cert-file="" Nov 03 21:44:42.279733 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271840 2578 flags.go:64] FLAG: --tls-cipher-suites="[]" Nov 03 21:44:42.279733 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271845 2578 flags.go:64] FLAG: --tls-min-version="" Nov 03 21:44:42.280381 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271848 2578 flags.go:64] FLAG: --tls-private-key-file="" Nov 03 21:44:42.280381 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271851 2578 flags.go:64] FLAG: --topology-manager-policy="none" Nov 03 21:44:42.280381 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271855 2578 flags.go:64] FLAG: --topology-manager-policy-options="" Nov 03 21:44:42.280381 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271859 2578 flags.go:64] FLAG: --topology-manager-scope="container" Nov 03 21:44:42.280381 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271863 2578 flags.go:64] FLAG: --v="2" Nov 03 21:44:42.280381 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271867 2578 flags.go:64] FLAG: --version="false" Nov 03 21:44:42.280381 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271872 2578 flags.go:64] FLAG: --vmodule="" Nov 03 21:44:42.280381 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271876 2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Nov 03 21:44:42.280381 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.271880 2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Nov 03 21:44:42.280381 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272032 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 03 21:44:42.280381 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272037 2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 03 21:44:42.280381 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272040 2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 03 21:44:42.280381 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272043 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 03 21:44:42.280381 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272046 2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 03 21:44:42.280381 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272049 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 03 21:44:42.280381 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272051 2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 03 21:44:42.280381 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272054 2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 03 21:44:42.280381 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272057 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 03 21:44:42.280381 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272060 2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 03 21:44:42.280381 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272063 2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 03 21:44:42.280381 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272065 2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 03 21:44:42.280884 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272068 2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 03 21:44:42.280884 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272070 2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 03 21:44:42.280884 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272073 2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 03 21:44:42.280884 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272075 2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 03 21:44:42.280884 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272078 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 03 21:44:42.280884 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272081 2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 03 21:44:42.280884 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272084 2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 03 21:44:42.280884 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272086 2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 03 21:44:42.280884 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272089 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 03 21:44:42.280884 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272091 2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 03 21:44:42.280884 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272094 2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 03 21:44:42.280884 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272111 2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 03 21:44:42.280884 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272114 2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 03 21:44:42.280884 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272118 2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 03 21:44:42.280884 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272122 2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 03 21:44:42.280884 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272126 2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 03 21:44:42.280884 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272129 2578 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 03 21:44:42.280884 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272132 2578 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 03 21:44:42.280884 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272135 2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 03 21:44:42.280884 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272137 2578 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 03 21:44:42.281472 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272140 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 03 21:44:42.281472 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272143 2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 03 21:44:42.281472 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272146 2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 03 21:44:42.281472 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272149 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 03 21:44:42.281472 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272151 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 03 21:44:42.281472 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272154 2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 03 21:44:42.281472 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272156 2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 03 21:44:42.281472 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272159 2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 03 21:44:42.281472 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272161 2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 03 21:44:42.281472 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272164 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 03 21:44:42.281472 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272166 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 03 21:44:42.281472 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272169 2578 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 03 21:44:42.281472 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272171 2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 03 21:44:42.281472 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272174 2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 03 21:44:42.281472 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272177 2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 03 21:44:42.281472 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272179 2578 feature_gate.go:386] unrecognized feature gate: Example Nov 03 21:44:42.281472 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272182 2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 03 21:44:42.281472 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272185 2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 03 21:44:42.281472 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272187 2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 03 21:44:42.281472 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272190 2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 03 21:44:42.281946 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272192 2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 03 21:44:42.281946 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272195 2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 03 21:44:42.281946 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272197 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 03 21:44:42.281946 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272200 2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 03 21:44:42.281946 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272202 2578 feature_gate.go:386] unrecognized feature gate: EVPN Nov 03 21:44:42.281946 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272205 2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 03 21:44:42.281946 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272208 2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 03 21:44:42.281946 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272212 2578 feature_gate.go:386] unrecognized feature gate: Example2 Nov 03 21:44:42.281946 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272215 2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 03 21:44:42.281946 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272218 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 03 21:44:42.281946 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272220 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 03 21:44:42.281946 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272223 2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 03 21:44:42.281946 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272226 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 03 21:44:42.281946 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272229 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 03 21:44:42.281946 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272231 2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 03 21:44:42.281946 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272234 2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 03 21:44:42.281946 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272236 2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 03 21:44:42.281946 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272239 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 03 21:44:42.281946 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272241 2578 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 03 21:44:42.281946 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272244 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 03 21:44:42.282451 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272246 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 03 21:44:42.282451 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272250 2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 03 21:44:42.282451 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272254 2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 03 21:44:42.282451 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272257 2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 03 21:44:42.282451 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272260 2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 03 21:44:42.282451 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272262 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 03 21:44:42.282451 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272265 2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 03 21:44:42.282451 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272267 2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 03 21:44:42.282451 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272270 2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 03 21:44:42.282451 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272272 2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 03 21:44:42.282451 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272275 2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 03 21:44:42.282451 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272278 2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 03 21:44:42.282451 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272280 2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 03 21:44:42.282451 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272283 2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 03 21:44:42.282451 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272285 2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 03 21:44:42.282451 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272288 2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 03 21:44:42.282451 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272290 2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 03 21:44:42.282451 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272293 2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 03 21:44:42.282451 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272295 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 03 21:44:42.282912 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272300 2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 03 21:44:42.282912 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272304 2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 03 21:44:42.282912 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272306 2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 03 21:44:42.282912 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272309 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 03 21:44:42.282912 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272311 2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 03 21:44:42.282912 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272314 2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 03 21:44:42.282912 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272316 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 03 21:44:42.282912 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272320 2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 03 21:44:42.282912 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272322 2578 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 03 21:44:42.282912 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272325 2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 03 21:44:42.282912 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272327 2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 03 21:44:42.282912 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272330 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 03 21:44:42.282912 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272333 2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 03 21:44:42.282912 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272335 2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 03 21:44:42.282912 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272338 2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 03 21:44:42.282912 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.272363 2578 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 03 21:44:42.283321 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272406 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 03 21:44:42.283321 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272411 2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 03 21:44:42.283321 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272414 2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 03 21:44:42.283321 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272416 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 03 21:44:42.283321 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272419 2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 03 21:44:42.283321 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272422 2578 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 03 21:44:42.283321 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272425 2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 03 21:44:42.283321 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272428 2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 03 21:44:42.283321 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272431 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 03 21:44:42.283321 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272433 2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 03 21:44:42.283321 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272436 2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 03 21:44:42.283321 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272439 2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 03 21:44:42.283321 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272442 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 03 21:44:42.283321 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272444 2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 03 21:44:42.283321 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272447 2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 03 21:44:42.283321 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272449 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 03 21:44:42.283321 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272453 2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 03 21:44:42.283321 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272455 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 03 21:44:42.283321 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272458 2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 03 21:44:42.283321 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272460 2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 03 21:44:42.283790 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272463 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 03 21:44:42.283790 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272466 2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 03 21:44:42.283790 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272468 2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 03 21:44:42.283790 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272471 2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 03 21:44:42.283790 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272474 2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 03 21:44:42.283790 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272477 2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 03 21:44:42.283790 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272479 2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 03 21:44:42.283790 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272491 2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 03 21:44:42.283790 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272496 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 03 21:44:42.283790 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272498 2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 03 21:44:42.283790 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272501 2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 03 21:44:42.283790 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272503 2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 03 21:44:42.283790 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272506 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 03 21:44:42.283790 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272508 2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 03 21:44:42.283790 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272511 2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 03 21:44:42.283790 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272513 2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 03 21:44:42.283790 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272516 2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 03 21:44:42.283790 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272519 2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 03 21:44:42.283790 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272521 2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 03 21:44:42.284269 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272524 2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 03 21:44:42.284269 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272527 2578 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 03 21:44:42.284269 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272529 2578 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 03 21:44:42.284269 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272532 2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 03 21:44:42.284269 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272534 2578 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 03 21:44:42.284269 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272537 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 03 21:44:42.284269 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272539 2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 03 21:44:42.284269 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272543 2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 03 21:44:42.284269 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272546 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 03 21:44:42.284269 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272548 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 03 21:44:42.284269 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272551 2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 03 21:44:42.284269 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272553 2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 03 21:44:42.284269 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272556 2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 03 21:44:42.284269 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272559 2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 03 21:44:42.284269 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272561 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 03 21:44:42.284269 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272564 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 03 21:44:42.284269 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272566 2578 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 03 21:44:42.284269 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272569 2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 03 21:44:42.284269 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272577 2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 03 21:44:42.284269 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272580 2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 03 21:44:42.284753 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272583 2578 feature_gate.go:386] unrecognized feature gate: Example Nov 03 21:44:42.284753 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272585 2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 03 21:44:42.284753 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272588 2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 03 21:44:42.284753 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272591 2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 03 21:44:42.284753 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272594 2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 03 21:44:42.284753 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272596 2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 03 21:44:42.284753 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272599 2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 03 21:44:42.284753 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272601 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 03 21:44:42.284753 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272604 2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 03 21:44:42.284753 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272607 2578 feature_gate.go:386] unrecognized feature gate: EVPN Nov 03 21:44:42.284753 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272609 2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 03 21:44:42.284753 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272612 2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 03 21:44:42.284753 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272614 2578 feature_gate.go:386] unrecognized feature gate: Example2 Nov 03 21:44:42.284753 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272618 2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 03 21:44:42.284753 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272620 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 03 21:44:42.284753 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272623 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 03 21:44:42.284753 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272625 2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 03 21:44:42.284753 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272628 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 03 21:44:42.284753 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272630 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 03 21:44:42.284753 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272634 2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 03 21:44:42.285288 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272637 2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 03 21:44:42.285288 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272639 2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 03 21:44:42.285288 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272642 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 03 21:44:42.285288 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272644 2578 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 03 21:44:42.285288 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272647 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 03 21:44:42.285288 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272649 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 03 21:44:42.285288 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272654 2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 03 21:44:42.285288 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272657 2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 03 21:44:42.285288 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272660 2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 03 21:44:42.285288 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272663 2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 03 21:44:42.285288 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272665 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 03 21:44:42.285288 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272675 2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 03 21:44:42.285288 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272678 2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 03 21:44:42.285288 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272680 2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 03 21:44:42.285288 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272683 2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 03 21:44:42.285288 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272686 2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 03 21:44:42.285288 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272689 2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 03 21:44:42.285288 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272691 2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 03 21:44:42.285288 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272694 2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 03 21:44:42.285727 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272696 2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 03 21:44:42.285727 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272699 2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 03 21:44:42.285727 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272701 2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 03 21:44:42.285727 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272704 2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 03 21:44:42.285727 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272706 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 03 21:44:42.285727 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272711 2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 03 21:44:42.285727 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272715 2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 03 21:44:42.285727 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.272718 2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 03 21:44:42.285727 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.279607 2578 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Nov 03 21:44:42.285727 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.279753 2578 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Nov 03 21:44:42.285727 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.280397 2578 watchdog_linux.go:95] "Systemd watchdog is not enabled" Nov 03 21:44:42.285727 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.280405 2578 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Nov 03 21:44:42.285727 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280493 2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 03 21:44:42.285727 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280500 2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 03 21:44:42.285727 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280503 2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 03 21:44:42.285727 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280506 2578 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 03 21:44:42.285727 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280509 2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 03 21:44:42.285727 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280512 2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 03 21:44:42.285727 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280514 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 03 21:44:42.286208 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280517 2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 03 21:44:42.286208 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280520 2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 03 21:44:42.286208 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280522 2578 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 03 21:44:42.286208 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280525 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 03 21:44:42.286208 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280528 2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 03 21:44:42.286208 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280531 2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 03 21:44:42.286208 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280534 2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 03 21:44:42.286208 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280537 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 03 21:44:42.286208 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280539 2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 03 21:44:42.286208 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280542 2578 feature_gate.go:386] unrecognized feature gate: EVPN Nov 03 21:44:42.286208 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280545 2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 03 21:44:42.286208 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280547 2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 03 21:44:42.286208 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280550 2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 03 21:44:42.286208 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280553 2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 03 21:44:42.286208 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280555 2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 03 21:44:42.286208 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280558 2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 03 21:44:42.286208 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280562 2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 03 21:44:42.286208 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280566 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 03 21:44:42.286208 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280569 2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 03 21:44:42.286813 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280571 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 03 21:44:42.286813 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280574 2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 03 21:44:42.286813 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280578 2578 feature_gate.go:386] unrecognized feature gate: Example Nov 03 21:44:42.286813 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280582 2578 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 03 21:44:42.286813 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280585 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 03 21:44:42.286813 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280588 2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 03 21:44:42.286813 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280592 2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 03 21:44:42.286813 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280596 2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 03 21:44:42.286813 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280599 2578 feature_gate.go:386] unrecognized feature gate: Example2 Nov 03 21:44:42.286813 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280602 2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 03 21:44:42.286813 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280604 2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 03 21:44:42.286813 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280607 2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 03 21:44:42.286813 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280610 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 03 21:44:42.286813 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280612 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 03 21:44:42.286813 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280615 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 03 21:44:42.286813 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280617 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 03 21:44:42.286813 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280620 2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 03 21:44:42.286813 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280623 2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 03 21:44:42.286813 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280626 2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 03 21:44:42.287581 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280628 2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 03 21:44:42.287581 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280631 2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 03 21:44:42.287581 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280634 2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 03 21:44:42.287581 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280636 2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 03 21:44:42.287581 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280639 2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 03 21:44:42.287581 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280641 2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 03 21:44:42.287581 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280644 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 03 21:44:42.287581 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280646 2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 03 21:44:42.287581 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280649 2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 03 21:44:42.287581 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280651 2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 03 21:44:42.287581 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280654 2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 03 21:44:42.287581 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280656 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 03 21:44:42.287581 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280661 2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 03 21:44:42.287581 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280663 2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 03 21:44:42.287581 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280666 2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 03 21:44:42.287581 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280669 2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 03 21:44:42.287581 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280672 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 03 21:44:42.287581 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280675 2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 03 21:44:42.287581 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280678 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 03 21:44:42.288318 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280681 2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 03 21:44:42.288318 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280683 2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 03 21:44:42.288318 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280685 2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 03 21:44:42.288318 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280688 2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 03 21:44:42.288318 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280691 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 03 21:44:42.288318 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280693 2578 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 03 21:44:42.288318 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280695 2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 03 21:44:42.288318 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280698 2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 03 21:44:42.288318 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280700 2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 03 21:44:42.288318 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280703 2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 03 21:44:42.288318 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280706 2578 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 03 21:44:42.288318 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280708 2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 03 21:44:42.288318 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280711 2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 03 21:44:42.288318 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280713 2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 03 21:44:42.288318 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280715 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 03 21:44:42.288318 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280718 2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 03 21:44:42.288318 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280721 2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 03 21:44:42.288318 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280723 2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 03 21:44:42.288318 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280726 2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 03 21:44:42.288318 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280728 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 03 21:44:42.288955 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280730 2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 03 21:44:42.288955 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280733 2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 03 21:44:42.288955 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280736 2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 03 21:44:42.288955 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280738 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 03 21:44:42.288955 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280741 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 03 21:44:42.288955 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280744 2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 03 21:44:42.288955 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280746 2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 03 21:44:42.288955 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280749 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 03 21:44:42.288955 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280752 2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 03 21:44:42.288955 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280755 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 03 21:44:42.288955 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280758 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 03 21:44:42.288955 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280761 2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 03 21:44:42.288955 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280764 2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 03 21:44:42.288955 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280766 2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 03 21:44:42.288955 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280769 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 03 21:44:42.288955 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280772 2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 03 21:44:42.288955 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280774 2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 03 21:44:42.288955 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280777 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 03 21:44:42.288955 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280779 2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 03 21:44:42.289503 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280781 2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 03 21:44:42.289503 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280784 2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 03 21:44:42.289503 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280786 2578 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 03 21:44:42.289503 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.280814 2578 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 03 21:44:42.289503 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.280997 2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 03 21:44:42.289503 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281004 2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 03 21:44:42.289503 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281007 2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 03 21:44:42.289503 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281009 2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 03 21:44:42.289503 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281012 2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 03 21:44:42.289503 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281015 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 03 21:44:42.289503 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281018 2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 03 21:44:42.289503 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281021 2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 03 21:44:42.289503 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281024 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 03 21:44:42.289503 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281026 2578 feature_gate.go:386] unrecognized feature gate: Example Nov 03 21:44:42.289503 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281029 2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 03 21:44:42.289503 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281032 2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 03 21:44:42.289929 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281034 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 03 21:44:42.289929 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281037 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 03 21:44:42.289929 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281041 2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 03 21:44:42.289929 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281044 2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 03 21:44:42.289929 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281047 2578 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 03 21:44:42.289929 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281050 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 03 21:44:42.289929 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281053 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 03 21:44:42.289929 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281056 2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 03 21:44:42.289929 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281059 2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 03 21:44:42.289929 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281062 2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 03 21:44:42.289929 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281065 2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 03 21:44:42.289929 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281067 2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 03 21:44:42.289929 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281070 2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 03 21:44:42.289929 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281073 2578 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 03 21:44:42.289929 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281075 2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 03 21:44:42.289929 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281078 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 03 21:44:42.289929 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281080 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 03 21:44:42.289929 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281083 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 03 21:44:42.289929 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281086 2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 03 21:44:42.290444 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281089 2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 03 21:44:42.290444 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281091 2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 03 21:44:42.290444 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281094 2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 03 21:44:42.290444 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281096 2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 03 21:44:42.290444 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281100 2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 03 21:44:42.290444 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281104 2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 03 21:44:42.290444 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281107 2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 03 21:44:42.290444 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281110 2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 03 21:44:42.290444 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281112 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 03 21:44:42.290444 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281116 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 03 21:44:42.290444 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281118 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 03 21:44:42.290444 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281121 2578 feature_gate.go:386] unrecognized feature gate: Example2 Nov 03 21:44:42.290444 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281123 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 03 21:44:42.290444 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281126 2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 03 21:44:42.290444 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281128 2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 03 21:44:42.290444 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281131 2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 03 21:44:42.290444 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281133 2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 03 21:44:42.290444 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281136 2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 03 21:44:42.290877 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281138 2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 03 21:44:42.290877 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281141 2578 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 03 21:44:42.290877 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281144 2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 03 21:44:42.290877 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281146 2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 03 21:44:42.290877 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281149 2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 03 21:44:42.290877 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281152 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 03 21:44:42.290877 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281155 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 03 21:44:42.290877 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281157 2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 03 21:44:42.290877 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281160 2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 03 21:44:42.290877 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281162 2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 03 21:44:42.290877 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281165 2578 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 03 21:44:42.290877 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281167 2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 03 21:44:42.290877 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281170 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 03 21:44:42.290877 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281172 2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 03 21:44:42.290877 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281175 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 03 21:44:42.290877 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281177 2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 03 21:44:42.290877 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281180 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 03 21:44:42.290877 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281182 2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 03 21:44:42.290877 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281185 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 03 21:44:42.290877 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281187 2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 03 21:44:42.291796 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281190 2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 03 21:44:42.291796 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281192 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 03 21:44:42.291796 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281195 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 03 21:44:42.291796 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281197 2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 03 21:44:42.291796 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281200 2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 03 21:44:42.291796 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281203 2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 03 21:44:42.291796 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281205 2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 03 21:44:42.291796 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281208 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 03 21:44:42.291796 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281210 2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 03 21:44:42.291796 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281213 2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 03 21:44:42.291796 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281215 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 03 21:44:42.291796 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281218 2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 03 21:44:42.291796 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281220 2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 03 21:44:42.291796 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281223 2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 03 21:44:42.291796 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281225 2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 03 21:44:42.291796 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281228 2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 03 21:44:42.291796 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281231 2578 feature_gate.go:386] unrecognized feature gate: EVPN Nov 03 21:44:42.291796 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281233 2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 03 21:44:42.291796 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281236 2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 03 21:44:42.291796 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281239 2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 03 21:44:42.292380 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281241 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 03 21:44:42.292380 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281244 2578 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 03 21:44:42.292380 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281246 2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 03 21:44:42.292380 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281249 2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 03 21:44:42.292380 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281251 2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 03 21:44:42.292380 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281253 2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 03 21:44:42.292380 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281256 2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 03 21:44:42.292380 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281258 2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 03 21:44:42.292380 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281261 2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 03 21:44:42.292380 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281264 2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 03 21:44:42.292380 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281266 2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 03 21:44:42.292380 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281269 2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 03 21:44:42.292380 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281271 2578 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 03 21:44:42.292380 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281274 2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 03 21:44:42.292380 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281277 2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 03 21:44:42.292380 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281279 2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 03 21:44:42.292380 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281282 2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 03 21:44:42.292863 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.281306 2578 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 03 21:44:42.292863 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281343 2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 03 21:44:42.292863 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281348 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 03 21:44:42.292863 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281351 2578 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 03 21:44:42.292863 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281354 2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 03 21:44:42.292863 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281356 2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 03 21:44:42.292863 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281359 2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 03 21:44:42.292863 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281361 2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 03 21:44:42.292863 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281364 2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 03 21:44:42.292863 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281367 2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 03 21:44:42.292863 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281369 2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 03 21:44:42.292863 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281372 2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 03 21:44:42.292863 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281375 2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 03 21:44:42.292863 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281377 2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 03 21:44:42.292863 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281380 2578 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 03 21:44:42.292863 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281383 2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 03 21:44:42.293384 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281386 2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 03 21:44:42.293384 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281389 2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 03 21:44:42.293384 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281391 2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 03 21:44:42.293384 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281394 2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 03 21:44:42.293384 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281396 2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 03 21:44:42.293384 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281398 2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 03 21:44:42.293384 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281401 2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 03 21:44:42.293384 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281403 2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 03 21:44:42.293384 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281406 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 03 21:44:42.293384 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281408 2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 03 21:44:42.293384 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281411 2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 03 21:44:42.293384 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281414 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 03 21:44:42.293384 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281416 2578 feature_gate.go:386] unrecognized feature gate: Example Nov 03 21:44:42.293384 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281419 2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 03 21:44:42.293384 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281421 2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 03 21:44:42.293384 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281424 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 03 21:44:42.293384 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281427 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 03 21:44:42.293384 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281430 2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 03 21:44:42.293384 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281434 2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 03 21:44:42.294047 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281437 2578 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 03 21:44:42.294047 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281440 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 03 21:44:42.294047 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281443 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 03 21:44:42.294047 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281445 2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 03 21:44:42.294047 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281448 2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 03 21:44:42.294047 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281450 2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 03 21:44:42.294047 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281453 2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 03 21:44:42.294047 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281455 2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 03 21:44:42.294047 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281458 2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 03 21:44:42.294047 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281460 2578 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 03 21:44:42.294047 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281463 2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 03 21:44:42.294047 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281465 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 03 21:44:42.294047 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281469 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 03 21:44:42.294047 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281472 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 03 21:44:42.294047 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281475 2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 03 21:44:42.294047 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281477 2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 03 21:44:42.294047 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281480 2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 03 21:44:42.294047 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281483 2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 03 21:44:42.294047 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281486 2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 03 21:44:42.294534 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281490 2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 03 21:44:42.294534 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281494 2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 03 21:44:42.294534 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281497 2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 03 21:44:42.294534 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281499 2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 03 21:44:42.294534 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281502 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 03 21:44:42.294534 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281504 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 03 21:44:42.294534 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281507 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 03 21:44:42.294534 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281509 2578 feature_gate.go:386] unrecognized feature gate: Example2 Nov 03 21:44:42.294534 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281512 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 03 21:44:42.294534 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281515 2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 03 21:44:42.294534 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281517 2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 03 21:44:42.294534 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281520 2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 03 21:44:42.294534 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281522 2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 03 21:44:42.294534 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281525 2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 03 21:44:42.294534 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281527 2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 03 21:44:42.294534 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281530 2578 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 03 21:44:42.294534 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281533 2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 03 21:44:42.294534 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281535 2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 03 21:44:42.294961 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281540 2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 03 21:44:42.294961 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281543 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 03 21:44:42.294961 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281545 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 03 21:44:42.294961 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281548 2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 03 21:44:42.294961 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281550 2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 03 21:44:42.294961 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281553 2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 03 21:44:42.294961 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281555 2578 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 03 21:44:42.294961 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281558 2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 03 21:44:42.294961 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281561 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 03 21:44:42.294961 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281564 2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 03 21:44:42.294961 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281566 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 03 21:44:42.294961 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281569 2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 03 21:44:42.294961 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281571 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 03 21:44:42.294961 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281574 2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 03 21:44:42.294961 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281577 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 03 21:44:42.294961 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281579 2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 03 21:44:42.294961 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281582 2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 03 21:44:42.294961 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281584 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 03 21:44:42.294961 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281587 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 03 21:44:42.295498 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281589 2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 03 21:44:42.295498 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281592 2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 03 21:44:42.295498 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281594 2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 03 21:44:42.295498 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281597 2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 03 21:44:42.295498 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281601 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 03 21:44:42.295498 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281604 2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 03 21:44:42.295498 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281607 2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 03 21:44:42.295498 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281609 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 03 21:44:42.295498 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281612 2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 03 21:44:42.295498 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281614 2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 03 21:44:42.295498 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281617 2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 03 21:44:42.295498 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281620 2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 03 21:44:42.295498 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281623 2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 03 21:44:42.295498 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281627 2578 feature_gate.go:386] unrecognized feature gate: EVPN Nov 03 21:44:42.295498 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281630 2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 03 21:44:42.295498 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:42.281633 2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 03 21:44:42.295498 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.281693 2578 server.go:957] "Client rotation is on, will bootstrap in background" Nov 03 21:44:42.295498 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.283697 2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Nov 03 21:44:42.295498 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.284589 2578 server.go:1014] "Starting client certificate rotation" Nov 03 21:44:42.295498 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.284709 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Nov 03 21:44:42.295968 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.284747 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Nov 03 21:44:42.295968 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.286901 2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 03 21:44:42.295968 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.289308 2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 03 21:44:42.299730 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.299706 2578 log.go:25] "Validated CRI v1 runtime API" Nov 03 21:44:42.302890 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.302873 2578 log.go:25] "Validated CRI v1 image API" Nov 03 21:44:42.303640 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.303618 2578 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Nov 03 21:44:42.306149 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.306128 2578 fs.go:135] Filesystem UUIDs: map[7141210b-eb02-42fb-9c77-7dd78df61112:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 8e1fc453-7c05-41e1-85fc-af7bbf0c43f3:/dev/nvme0n1p3] Nov 03 21:44:42.306210 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.306148 2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}] Nov 03 21:44:42.312229 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.312112 2578 manager.go:217] Machine: {Timestamp:2026-11-03 21:44:42.310808867 +0000 UTC m=+0.310176630 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3105654 MemoryCapacity:33164324864 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec29333d281fee1b3e0524a91d19a59e SystemUUID:ec29333d-281f-ee1b-3e05-24a91d19a59e BootID:156f93a1-8311-43bd-a0df-1877bda020d8 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582164480 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582160384 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632865792 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:d2:0e:1b:c7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:d2:0e:1b:c7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f2:c4:96:0d:9b:7c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164324864 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Nov 03 21:44:42.312229 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.312229 2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Nov 03 21:44:42.312362 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.312349 2578 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Nov 03 21:44:42.314056 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.314030 2578 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Nov 03 21:44:42.314200 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.314058 2578 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-7.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Nov 03 21:44:42.314244 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.314203 2578 topology_manager.go:143] "Creating topology manager with none policy" Nov 03 21:44:42.314244 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.314211 2578 container_manager_linux.go:311] "Creating device plugin manager" Nov 03 21:44:42.314297 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.314245 2578 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Nov 03 21:44:42.314297 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.314264 2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Nov 03 21:44:42.314998 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.314975 2578 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Nov 03 21:44:42.318877 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.318863 2578 state_mem.go:41] "Initialized" logger="CPUManager state memory" Nov 03 21:44:42.319186 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.319176 2578 server.go:1235] "Using root directory" path="/var/lib/kubelet" Nov 03 21:44:42.319228 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.319210 2578 kubelet.go:493] "Attempting to sync node with API server" Nov 03 21:44:42.319228 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.319222 2578 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Nov 03 21:44:42.319284 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.319232 2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Nov 03 21:44:42.319284 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.319242 2578 kubelet.go:405] "Adding apiserver pod source" Nov 03 21:44:42.319284 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.319249 2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Nov 03 21:44:42.319645 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.319633 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Nov 03 21:44:42.322004 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.321976 2578 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Nov 03 21:44:42.323765 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.323722 2578 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Nov 03 21:44:42.327695 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.327679 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Nov 03 21:44:42.328453 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.328440 2578 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Nov 03 21:44:42.328512 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.328471 2578 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Nov 03 21:44:42.328643 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.328632 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Nov 03 21:44:42.328676 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.328648 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Nov 03 21:44:42.328676 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.328655 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Nov 03 21:44:42.328676 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.328661 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Nov 03 21:44:42.328676 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.328666 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Nov 03 21:44:42.328676 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.328672 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Nov 03 21:44:42.329256 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.329245 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Nov 03 21:44:42.329289 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.329260 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Nov 03 21:44:42.329289 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.329272 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Nov 03 21:44:42.329289 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.329278 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Nov 03 21:44:42.329289 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.329287 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Nov 03 21:44:42.329419 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.329296 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Nov 03 21:44:42.330538 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.330527 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Nov 03 21:44:42.330538 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.330537 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Nov 03 21:44:42.334426 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.334413 2578 server.go:1263] "Started kubelet" Nov 03 21:44:42.334610 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.334550 2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Nov 03 21:44:42.334707 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.334569 2578 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Nov 03 21:44:42.334707 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.334628 2578 server_v1.go:49] "podresources" method="list" useActivePods=true Nov 03 21:44:42.335246 ip-10-0-137-7 systemd[1]: Started Kubernetes Kubelet. Nov 03 21:44:42.335633 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.335616 2578 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Nov 03 21:44:42.336154 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.336114 2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-7.us-east-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Nov 03 21:44:42.336402 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.336387 2578 server.go:317] "Adding debug handlers to kubelet server" Nov 03 21:44:42.341315 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:42.341288 2578 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Nov 03 21:44:42.341449 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.341429 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Nov 03 21:44:42.341924 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.341907 2578 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Nov 03 21:44:42.342059 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:42.342036 2578 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-137-7.us-east-2.compute.internal\" not found" Nov 03 21:44:42.342452 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.342436 2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Nov 03 21:44:42.342541 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.342439 2578 volume_manager.go:309] "The desired_state_of_world populator starts" Nov 03 21:44:42.342614 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.342606 2578 volume_manager.go:311] "Starting Kubelet Volume Manager" Nov 03 21:44:42.342789 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.342768 2578 factory.go:55] Registering systemd factory Nov 03 21:44:42.342876 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.342793 2578 factory.go:223] Registration of the systemd container factory successfully Nov 03 21:44:42.342876 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.342804 2578 reconstruct.go:97] "Volume reconstruction finished" Nov 03 21:44:42.342876 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.342817 2578 reconciler.go:29] "Reconciler: start to sync state" Nov 03 21:44:42.343060 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.343046 2578 factory.go:153] Registering CRI-O factory Nov 03 21:44:42.343124 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.343064 2578 factory.go:223] Registration of the crio container factory successfully Nov 03 21:44:42.343124 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.343116 2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Nov 03 21:44:42.343225 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.343135 2578 factory.go:103] Registering Raw factory Nov 03 21:44:42.343225 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.343151 2578 manager.go:1199] Started watching for new ooms in manager Nov 03 21:44:42.343932 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.343918 2578 manager.go:319] Starting recovery of all containers Nov 03 21:44:42.347171 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:42.347145 2578 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-7.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Nov 03 21:44:42.347774 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:42.347752 2578 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-7.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node" Nov 03 21:44:42.347872 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:42.347769 2578 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" Nov 03 21:44:42.349044 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:42.347872 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-7.us-east-2.compute.internal.18e4a71a2523b4d2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-7.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-7.us-east-2.compute.internal,},FirstTimestamp:2026-11-03 21:44:42.334385362 +0000 UTC m=+0.333753126,LastTimestamp:2026-11-03 21:44:42.334385362 +0000 UTC m=+0.333753126,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-7.us-east-2.compute.internal,}" Nov 03 21:44:42.349524 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.349497 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wqvql" Nov 03 21:44:42.354125 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:42.354095 2578 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" Nov 03 21:44:42.356852 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.356836 2578 manager.go:324] Recovery completed Nov 03 21:44:42.360448 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.360430 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wqvql" Nov 03 21:44:42.361631 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.361615 2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 03 21:44:42.363935 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.363916 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-7.us-east-2.compute.internal" event="NodeHasSufficientMemory" Nov 03 21:44:42.364026 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.363950 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-7.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Nov 03 21:44:42.364026 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.363966 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-7.us-east-2.compute.internal" event="NodeHasSufficientPID" Nov 03 21:44:42.364722 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.364705 2578 cpu_manager.go:226] "Starting" policy="none" Nov 03 21:44:42.364722 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.364718 2578 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Nov 03 21:44:42.364844 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.364734 2578 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Nov 03 21:44:42.364844 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.364761 2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 03 21:44:42.365730 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.365715 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-7.us-east-2.compute.internal" event="NodeHasSufficientMemory" Nov 03 21:44:42.365810 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.365741 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-7.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Nov 03 21:44:42.365810 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.365752 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-7.us-east-2.compute.internal" event="NodeHasSufficientPID" Nov 03 21:44:42.366298 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:42.366228 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-7.us-east-2.compute.internal.18e4a71a26e69192 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-7.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-7.us-east-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-7.us-east-2.compute.internal,},FirstTimestamp:2026-11-03 21:44:42.363933074 +0000 UTC m=+0.363300838,LastTimestamp:2026-11-03 21:44:42.363933074 +0000 UTC m=+0.363300838,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-7.us-east-2.compute.internal,}" Nov 03 21:44:42.443195 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:42.443158 2578 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-137-7.us-east-2.compute.internal\" not found" Nov 03 21:44:42.507527 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.507504 2578 policy_none.go:50] "Start" Nov 03 21:44:42.507527 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.507530 2578 memory_manager.go:187] "Starting memorymanager" policy="None" Nov 03 21:44:42.507627 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.507544 2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Nov 03 21:44:42.509790 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.509779 2578 policy_none.go:44] "Start" Nov 03 21:44:42.533602 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.533564 2578 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Nov 03 21:44:42.536121 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.536107 2578 manager.go:341] "Starting Device Plugin manager" Nov 03 21:44:42.536197 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:42.536138 2578 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Nov 03 21:44:42.536197 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.536148 2578 server.go:87] "Starting device plugin registration server" Nov 03 21:44:42.536469 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.536457 2578 eviction_manager.go:194] "Eviction manager: starting control loop" Nov 03 21:44:42.536508 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.536472 2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Nov 03 21:44:42.536577 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.536559 2578 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Nov 03 21:44:42.536662 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.536647 2578 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Nov 03 21:44:42.536662 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.536659 2578 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Nov 03 21:44:42.537192 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:42.537172 2578 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Nov 03 21:44:42.537263 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:42.537227 2578 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-7.us-east-2.compute.internal\" not found" Nov 03 21:44:42.554281 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:42.554260 2578 nodelease.go:50] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-137-7.us-east-2.compute.internal\" not found" node="ip-10-0-137-7.us-east-2.compute.internal" Nov 03 21:44:42.637281 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.637237 2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 03 21:44:42.639376 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.638431 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-7.us-east-2.compute.internal" event="NodeHasSufficientMemory" Nov 03 21:44:42.639376 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.638458 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-7.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Nov 03 21:44:42.639376 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.638471 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-7.us-east-2.compute.internal" event="NodeHasSufficientPID" Nov 03 21:44:42.639376 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.638501 2578 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-137-7.us-east-2.compute.internal" Nov 03 21:44:42.645345 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.645322 2578 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-137-7.us-east-2.compute.internal" Nov 03 21:44:42.645345 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:42.645345 2578 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-7.us-east-2.compute.internal\": node \"ip-10-0-137-7.us-east-2.compute.internal\" not found" Nov 03 21:44:42.680238 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.680201 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Nov 03 21:44:42.681433 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.681415 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Nov 03 21:44:42.681433 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.681441 2578 status_manager.go:249] "Starting to sync pod status with apiserver" Nov 03 21:44:42.681587 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.681465 2578 kubelet.go:2521] "Starting kubelet main sync loop" Nov 03 21:44:42.681587 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:42.681509 2578 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Nov 03 21:44:42.697561 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.697531 2578 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Nov 03 21:44:42.698671 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:42.698651 2578 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-137-7.us-east-2.compute.internal\" not found" Nov 03 21:44:42.782499 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.782416 2578 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-7.us-east-2.compute.internal"] Nov 03 21:44:42.782580 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.782506 2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 03 21:44:42.783532 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.783514 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-7.us-east-2.compute.internal" event="NodeHasSufficientMemory" Nov 03 21:44:42.783649 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.783548 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-7.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Nov 03 21:44:42.783649 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.783562 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-7.us-east-2.compute.internal" event="NodeHasSufficientPID" Nov 03 21:44:42.785807 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.785790 2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 03 21:44:42.786015 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.786000 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal" Nov 03 21:44:42.786068 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.786041 2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 03 21:44:42.786593 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.786574 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-7.us-east-2.compute.internal" event="NodeHasSufficientMemory" Nov 03 21:44:42.786676 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.786615 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-7.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Nov 03 21:44:42.786676 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.786626 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-7.us-east-2.compute.internal" event="NodeHasSufficientPID" Nov 03 21:44:42.786676 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.786577 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-7.us-east-2.compute.internal" event="NodeHasSufficientMemory" Nov 03 21:44:42.786774 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.786684 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-7.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Nov 03 21:44:42.786774 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.786699 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-7.us-east-2.compute.internal" event="NodeHasSufficientPID" Nov 03 21:44:42.788748 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.788734 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-7.us-east-2.compute.internal" Nov 03 21:44:42.788813 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.788759 2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 03 21:44:42.789452 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.789438 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-7.us-east-2.compute.internal" event="NodeHasSufficientMemory" Nov 03 21:44:42.789536 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.789462 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-7.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Nov 03 21:44:42.789536 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.789472 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-7.us-east-2.compute.internal" event="NodeHasSufficientPID" Nov 03 21:44:42.798738 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:42.798721 2578 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-137-7.us-east-2.compute.internal\" not found" Nov 03 21:44:42.819700 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:42.819678 2578 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-7.us-east-2.compute.internal\" not found" node="ip-10-0-137-7.us-east-2.compute.internal" Nov 03 21:44:42.822452 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:42.822436 2578 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-7.us-east-2.compute.internal\" not found" node="ip-10-0-137-7.us-east-2.compute.internal" Nov 03 21:44:42.844524 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.844497 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/58ba030777ca53e38084de26650a107e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal\" (UID: \"58ba030777ca53e38084de26650a107e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal" Nov 03 21:44:42.844635 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.844539 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/58ba030777ca53e38084de26650a107e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal\" (UID: \"58ba030777ca53e38084de26650a107e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal" Nov 03 21:44:42.844635 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.844560 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/70b8abbfbbbc47301b7da80ca6b44429-config\") pod \"kube-apiserver-proxy-ip-10-0-137-7.us-east-2.compute.internal\" (UID: \"70b8abbfbbbc47301b7da80ca6b44429\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-7.us-east-2.compute.internal" Nov 03 21:44:42.899068 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:42.899039 2578 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-137-7.us-east-2.compute.internal\" not found" Nov 03 21:44:42.945464 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.945436 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/58ba030777ca53e38084de26650a107e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal\" (UID: \"58ba030777ca53e38084de26650a107e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal" Nov 03 21:44:42.945544 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.945488 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/58ba030777ca53e38084de26650a107e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal\" (UID: \"58ba030777ca53e38084de26650a107e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal" Nov 03 21:44:42.945544 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.945443 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/58ba030777ca53e38084de26650a107e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal\" (UID: \"58ba030777ca53e38084de26650a107e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal" Nov 03 21:44:42.945544 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.945519 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/70b8abbfbbbc47301b7da80ca6b44429-config\") pod \"kube-apiserver-proxy-ip-10-0-137-7.us-east-2.compute.internal\" (UID: \"70b8abbfbbbc47301b7da80ca6b44429\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-7.us-east-2.compute.internal" Nov 03 21:44:42.945668 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.945565 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/58ba030777ca53e38084de26650a107e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal\" (UID: \"58ba030777ca53e38084de26650a107e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal" Nov 03 21:44:42.945668 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:42.945572 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/70b8abbfbbbc47301b7da80ca6b44429-config\") pod \"kube-apiserver-proxy-ip-10-0-137-7.us-east-2.compute.internal\" (UID: \"70b8abbfbbbc47301b7da80ca6b44429\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-7.us-east-2.compute.internal" Nov 03 21:44:42.999574 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:42.999546 2578 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-137-7.us-east-2.compute.internal\" not found" Nov 03 21:44:43.100089 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:43.100034 2578 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-137-7.us-east-2.compute.internal\" not found" Nov 03 21:44:43.123243 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:43.123218 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal" Nov 03 21:44:43.125394 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:43.125376 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-7.us-east-2.compute.internal" Nov 03 21:44:44.890350 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:44.890323 2578 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-137-7.us-east-2.compute.internal\" not found" Nov 03 21:44:44.894448 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:44.894432 2578 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Nov 03 21:44:44.923397 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:44.923372 2578 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Nov 03 21:44:44.932586 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:44.932562 2578 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal" Nov 03 21:44:44.945271 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:44.945254 2578 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 03 21:44:44.946961 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:44.946947 2578 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-7.us-east-2.compute.internal" Nov 03 21:44:44.954317 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:44.954301 2578 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 03 21:44:44.974497 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:44.974473 2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Nov 03 21:44:44.974904 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:44.974634 2578 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received" Nov 03 21:44:44.974904 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:44.974667 2578 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received" Nov 03 21:44:44.974904 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:44.974658 2578 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received" Nov 03 21:44:45.009924 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.009895 2578 apiserver.go:52] "Watching apiserver" Nov 03 21:44:45.028694 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.028670 2578 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Nov 03 21:44:45.028959 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.028942 2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-9pctw","openshift-multus/multus-additional-cni-plugins-4cz4g","openshift-multus/multus-mbksx","openshift-multus/network-metrics-daemon-knt57","kube-system/kube-apiserver-proxy-ip-10-0-137-7.us-east-2.compute.internal","openshift-network-diagnostics/network-check-target-zrbsd","openshift-ovn-kubernetes/ovnkube-node-x26mj","kube-system/global-pull-secret-syncer-jq6hj","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c","openshift-cluster-node-tuning-operator/tuned-xw7xr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal","kube-system/konnectivity-agent-v98zn","openshift-image-registry/node-ca-nz4zw","openshift-network-operator/iptables-alerter-nkngm"] Nov 03 21:44:45.031482 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.031465 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9pctw" Nov 03 21:44:45.031683 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.031666 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Nov 03 21:44:45.033481 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.033464 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4cz4g" Nov 03 21:44:45.037167 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.035395 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Nov 03 21:44:45.037167 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.035897 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Nov 03 21:44:45.037167 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.036032 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-jbpmc\"" Nov 03 21:44:45.037582 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.037266 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.037641 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.037584 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c-system-cni-dir\") pod \"multus-additional-cni-plugins-4cz4g\" (UID: \"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c\") " pod="openshift-multus/multus-additional-cni-plugins-4cz4g" Nov 03 21:44:45.037641 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.037620 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c-cnibin\") pod \"multus-additional-cni-plugins-4cz4g\" (UID: \"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c\") " pod="openshift-multus/multus-additional-cni-plugins-4cz4g" Nov 03 21:44:45.037744 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.037649 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c-cni-binary-copy\") pod \"multus-additional-cni-plugins-4cz4g\" (UID: \"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c\") " pod="openshift-multus/multus-additional-cni-plugins-4cz4g" Nov 03 21:44:45.037744 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.037675 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4cz4g\" (UID: \"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c\") " pod="openshift-multus/multus-additional-cni-plugins-4cz4g" Nov 03 21:44:45.037744 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.037702 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4cz4g\" (UID: \"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c\") " pod="openshift-multus/multus-additional-cni-plugins-4cz4g" Nov 03 21:44:45.037744 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.037731 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9f5c183e-d58a-4621-940c-04a7ae0bafc9-hosts-file\") pod \"node-resolver-9pctw\" (UID: \"9f5c183e-d58a-4621-940c-04a7ae0bafc9\") " pod="openshift-dns/node-resolver-9pctw" Nov 03 21:44:45.037909 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.037754 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f5c183e-d58a-4621-940c-04a7ae0bafc9-tmp-dir\") pod \"node-resolver-9pctw\" (UID: \"9f5c183e-d58a-4621-940c-04a7ae0bafc9\") " pod="openshift-dns/node-resolver-9pctw" Nov 03 21:44:45.037909 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.037780 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rg4r2\" (UniqueName: \"kubernetes.io/projected/9f5c183e-d58a-4621-940c-04a7ae0bafc9-kube-api-access-rg4r2\") pod \"node-resolver-9pctw\" (UID: \"9f5c183e-d58a-4621-940c-04a7ae0bafc9\") " pod="openshift-dns/node-resolver-9pctw" Nov 03 21:44:45.037909 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.037848 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4cz4g\" (UID: \"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c\") " pod="openshift-multus/multus-additional-cni-plugins-4cz4g" Nov 03 21:44:45.038011 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.037902 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csdhx\" (UniqueName: \"kubernetes.io/projected/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c-kube-api-access-csdhx\") pod \"multus-additional-cni-plugins-4cz4g\" (UID: \"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c\") " pod="openshift-multus/multus-additional-cni-plugins-4cz4g" Nov 03 21:44:45.039786 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.039772 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knt57" Nov 03 21:44:45.039856 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:45.039840 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-knt57" podUID="eb5f720e-f465-439f-ba32-4f595561f9f6" Nov 03 21:44:45.041529 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.041493 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Nov 03 21:44:45.041819 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.041801 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrbsd" Nov 03 21:44:45.041902 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:45.041851 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrbsd" podUID="5d7e7316-1bd4-42cf-8519-544c1e2a4526" Nov 03 21:44:45.044010 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.043996 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.046388 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.046369 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:44:45.046476 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:45.046438 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jq6hj" podUID="94aaa3ac-c7fc-4f87-878e-8ef79127488b" Nov 03 21:44:45.048626 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.048608 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" Nov 03 21:44:45.050875 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.050860 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.053084 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.053068 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-v98zn" Nov 03 21:44:45.055293 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.055256 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-02 21:39:44 +0000 UTC" deadline="2028-06-16 03:23:46.526198818 +0000 UTC" Nov 03 21:44:45.055293 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.055282 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nz4zw" Nov 03 21:44:45.055436 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.055289 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14165h39m1.470912032s" Nov 03 21:44:45.057582 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.057565 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nkngm" Nov 03 21:44:45.069008 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.068993 2578 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Nov 03 21:44:45.069067 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.069032 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-c4gn4" Nov 03 21:44:45.080243 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.080221 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-c4gn4" Nov 03 21:44:45.133916 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.133894 2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Nov 03 21:44:45.136360 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.136328 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Nov 03 21:44:45.136481 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.136464 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Nov 03 21:44:45.136674 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.136658 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Nov 03 21:44:45.136738 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.136703 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Nov 03 21:44:45.136786 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.136740 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Nov 03 21:44:45.136828 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.136787 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-rdd92\"" Nov 03 21:44:45.136828 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.136804 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Nov 03 21:44:45.136900 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.136832 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-lbb62\"" Nov 03 21:44:45.136946 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.136936 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Nov 03 21:44:45.136984 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.136953 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Nov 03 21:44:45.137105 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.137092 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Nov 03 21:44:45.137228 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.137214 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Nov 03 21:44:45.137290 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.137236 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Nov 03 21:44:45.137290 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.137254 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Nov 03 21:44:45.137290 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.137254 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Nov 03 21:44:45.137290 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.137281 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Nov 03 21:44:45.137290 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.137281 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Nov 03 21:44:45.137449 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.137284 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Nov 03 21:44:45.138175 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.138158 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0db6d946-a1d8-4459-a883-6f93f7e1182a-iptables-alerter-script\") pod \"iptables-alerter-nkngm\" (UID: \"0db6d946-a1d8-4459-a883-6f93f7e1182a\") " pod="openshift-network-operator/iptables-alerter-nkngm" Nov 03 21:44:45.138254 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.138188 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-cnibin\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.138254 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.138236 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-hostroot\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.138356 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.138269 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c-system-cni-dir\") pod \"multus-additional-cni-plugins-4cz4g\" (UID: \"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c\") " pod="openshift-multus/multus-additional-cni-plugins-4cz4g" Nov 03 21:44:45.138356 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.138314 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Nov 03 21:44:45.138356 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.138316 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/94aaa3ac-c7fc-4f87-878e-8ef79127488b-kubelet-config\") pod \"global-pull-secret-syncer-jq6hj\" (UID: \"94aaa3ac-c7fc-4f87-878e-8ef79127488b\") " pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:44:45.138356 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.138329 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Nov 03 21:44:45.138356 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.138337 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-fc4bx\"" Nov 03 21:44:45.138356 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.138355 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Nov 03 21:44:45.138594 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.138357 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Nov 03 21:44:45.138594 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.138366 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Nov 03 21:44:45.138594 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.138378 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c-system-cni-dir\") pod \"multus-additional-cni-plugins-4cz4g\" (UID: \"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c\") " pod="openshift-multus/multus-additional-cni-plugins-4cz4g" Nov 03 21:44:45.138594 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.138394 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-hgwjm\"" Nov 03 21:44:45.138594 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.138399 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Nov 03 21:44:45.138594 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.138408 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Nov 03 21:44:45.138594 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.138360 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Nov 03 21:44:45.138594 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.138423 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-txfgg\"" Nov 03 21:44:45.138594 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.138435 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl5nr\" (UniqueName: \"kubernetes.io/projected/219625cd-f1a4-4e51-90ee-58f0fc5939bb-kube-api-access-kl5nr\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.138594 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.138436 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-lqk2r\"" Nov 03 21:44:45.138594 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.138473 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/48d8b783-b602-47ef-9b9b-f2416d448908-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wvd2c\" (UID: \"48d8b783-b602-47ef-9b9b-f2416d448908\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" Nov 03 21:44:45.138594 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.138493 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-52n7c\"" Nov 03 21:44:45.138594 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.138498 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-host-var-lib-cni-bin\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.138594 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.138538 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-bc4mv\"" Nov 03 21:44:45.138594 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.138546 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/48d8b783-b602-47ef-9b9b-f2416d448908-socket-dir\") pod \"aws-ebs-csi-driver-node-wvd2c\" (UID: \"48d8b783-b602-47ef-9b9b-f2416d448908\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" Nov 03 21:44:45.138594 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.138574 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/48d8b783-b602-47ef-9b9b-f2416d448908-device-dir\") pod \"aws-ebs-csi-driver-node-wvd2c\" (UID: \"48d8b783-b602-47ef-9b9b-f2416d448908\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" Nov 03 21:44:45.138594 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.138599 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-etc-sysctl-conf\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.139201 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.138630 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-etc-systemd\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.139201 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.138693 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-host-cni-netd\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.139201 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.138713 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.139201 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.138730 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/94aaa3ac-c7fc-4f87-878e-8ef79127488b-dbus\") pod \"global-pull-secret-syncer-jq6hj\" (UID: \"94aaa3ac-c7fc-4f87-878e-8ef79127488b\") " pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:44:45.139201 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.138746 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6c6w8\" (UniqueName: \"kubernetes.io/projected/0db6d946-a1d8-4459-a883-6f93f7e1182a-kube-api-access-6c6w8\") pod \"iptables-alerter-nkngm\" (UID: \"0db6d946-a1d8-4459-a883-6f93f7e1182a\") " pod="openshift-network-operator/iptables-alerter-nkngm" Nov 03 21:44:45.139201 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.138777 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-host-run-k8s-cni-cncf-io\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.139201 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.138798 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-systemd-units\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.139201 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.138834 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-run-systemd\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.139201 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.138867 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmr4b\" (UniqueName: \"kubernetes.io/projected/76122905-15b4-4f4e-97b9-9f73a7675e36-kube-api-access-tmr4b\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.139201 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.138886 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-host-run-ovn-kubernetes\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.139201 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.138905 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/811514f5-baff-4fc1-9e92-28f27de7b0b7-agent-certs\") pod \"konnectivity-agent-v98zn\" (UID: \"811514f5-baff-4fc1-9e92-28f27de7b0b7\") " pod="kube-system/konnectivity-agent-v98zn" Nov 03 21:44:45.139201 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.138941 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-multus-conf-dir\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.139201 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.138957 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb5f720e-f465-439f-ba32-4f595561f9f6-metrics-certs\") pod \"network-metrics-daemon-knt57\" (UID: \"eb5f720e-f465-439f-ba32-4f595561f9f6\") " pod="openshift-multus/network-metrics-daemon-knt57" Nov 03 21:44:45.139201 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.139019 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-var-lib-kubelet\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.139201 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.139050 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9f5c183e-d58a-4621-940c-04a7ae0bafc9-hosts-file\") pod \"node-resolver-9pctw\" (UID: \"9f5c183e-d58a-4621-940c-04a7ae0bafc9\") " pod="openshift-dns/node-resolver-9pctw" Nov 03 21:44:45.139201 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.139097 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-multus-socket-dir-parent\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.139734 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.139115 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9f5c183e-d58a-4621-940c-04a7ae0bafc9-hosts-file\") pod \"node-resolver-9pctw\" (UID: \"9f5c183e-d58a-4621-940c-04a7ae0bafc9\") " pod="openshift-dns/node-resolver-9pctw" Nov 03 21:44:45.139734 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.139148 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-host-run-multus-certs\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.139734 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.139172 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-node-log\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.139734 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.139186 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/219625cd-f1a4-4e51-90ee-58f0fc5939bb-cni-binary-copy\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.139734 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.139208 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-host-var-lib-kubelet\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.139734 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.139234 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8fqs\" (UniqueName: \"kubernetes.io/projected/48d8b783-b602-47ef-9b9b-f2416d448908-kube-api-access-k8fqs\") pod \"aws-ebs-csi-driver-node-wvd2c\" (UID: \"48d8b783-b602-47ef-9b9b-f2416d448908\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" Nov 03 21:44:45.139734 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.139252 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-etc-sysconfig\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.139734 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.139276 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4cz4g\" (UID: \"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c\") " pod="openshift-multus/multus-additional-cni-plugins-4cz4g" Nov 03 21:44:45.139734 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.139292 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-system-cni-dir\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.139734 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.139321 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-lib-modules\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.139734 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.139342 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0db6d946-a1d8-4459-a883-6f93f7e1182a-host-slash\") pod \"iptables-alerter-nkngm\" (UID: \"0db6d946-a1d8-4459-a883-6f93f7e1182a\") " pod="openshift-network-operator/iptables-alerter-nkngm" Nov 03 21:44:45.139734 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.139369 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c-cni-binary-copy\") pod \"multus-additional-cni-plugins-4cz4g\" (UID: \"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c\") " pod="openshift-multus/multus-additional-cni-plugins-4cz4g" Nov 03 21:44:45.139734 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.139392 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4cz4g\" (UID: \"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c\") " pod="openshift-multus/multus-additional-cni-plugins-4cz4g" Nov 03 21:44:45.139734 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.139497 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/76122905-15b4-4f4e-97b9-9f73a7675e36-ovn-node-metrics-cert\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.139734 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.139527 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a440580e-b0bd-49e2-a33d-576118e049ae-serviceca\") pod \"node-ca-nz4zw\" (UID: \"a440580e-b0bd-49e2-a33d-576118e049ae\") " pod="openshift-image-registry/node-ca-nz4zw" Nov 03 21:44:45.139734 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.139561 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-var-lib-openvswitch\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.140435 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.139586 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4cz4g\" (UID: \"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c\") " pod="openshift-multus/multus-additional-cni-plugins-4cz4g" Nov 03 21:44:45.140435 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.139601 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/76122905-15b4-4f4e-97b9-9f73a7675e36-ovnkube-config\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.140435 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.139633 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a440580e-b0bd-49e2-a33d-576118e049ae-host\") pod \"node-ca-nz4zw\" (UID: \"a440580e-b0bd-49e2-a33d-576118e049ae\") " pod="openshift-image-registry/node-ca-nz4zw" Nov 03 21:44:45.140435 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.139658 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqplq\" (UniqueName: \"kubernetes.io/projected/a440580e-b0bd-49e2-a33d-576118e049ae-kube-api-access-lqplq\") pod \"node-ca-nz4zw\" (UID: \"a440580e-b0bd-49e2-a33d-576118e049ae\") " pod="openshift-image-registry/node-ca-nz4zw" Nov 03 21:44:45.140435 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.139679 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/48d8b783-b602-47ef-9b9b-f2416d448908-registration-dir\") pod \"aws-ebs-csi-driver-node-wvd2c\" (UID: \"48d8b783-b602-47ef-9b9b-f2416d448908\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" Nov 03 21:44:45.140435 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.139701 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-sys\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.140435 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.139730 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csdhx\" (UniqueName: \"kubernetes.io/projected/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c-kube-api-access-csdhx\") pod \"multus-additional-cni-plugins-4cz4g\" (UID: \"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c\") " pod="openshift-multus/multus-additional-cni-plugins-4cz4g" Nov 03 21:44:45.140435 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.139771 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-host-slash\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.140435 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.139867 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-host-run-netns\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.140435 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.139896 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4cz4g\" (UID: \"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c\") " pod="openshift-multus/multus-additional-cni-plugins-4cz4g" Nov 03 21:44:45.140435 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.139923 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c-cni-binary-copy\") pod \"multus-additional-cni-plugins-4cz4g\" (UID: \"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c\") " pod="openshift-multus/multus-additional-cni-plugins-4cz4g" Nov 03 21:44:45.140435 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.139934 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-etc-openvswitch\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.140435 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.139961 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zh4kw\" (UniqueName: \"kubernetes.io/projected/5d7e7316-1bd4-42cf-8519-544c1e2a4526-kube-api-access-zh4kw\") pod \"network-check-target-zrbsd\" (UID: \"5d7e7316-1bd4-42cf-8519-544c1e2a4526\") " pod="openshift-network-diagnostics/network-check-target-zrbsd" Nov 03 21:44:45.140435 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.139979 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/94aaa3ac-c7fc-4f87-878e-8ef79127488b-original-pull-secret\") pod \"global-pull-secret-syncer-jq6hj\" (UID: \"94aaa3ac-c7fc-4f87-878e-8ef79127488b\") " pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:44:45.140435 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.140028 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-multus-cni-dir\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.140435 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.140063 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-host-run-netns\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.141060 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.140108 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-host-var-lib-cni-multus\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.141060 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.140156 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/48d8b783-b602-47ef-9b9b-f2416d448908-etc-selinux\") pod \"aws-ebs-csi-driver-node-wvd2c\" (UID: \"48d8b783-b602-47ef-9b9b-f2416d448908\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" Nov 03 21:44:45.141060 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.140182 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-host\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.141060 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.140215 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9k8xx\" (UniqueName: \"kubernetes.io/projected/9c970a61-3ebf-48c9-a05b-688accd0b18a-kube-api-access-9k8xx\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.141060 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.140233 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-log-socket\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.141060 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.140263 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c-cnibin\") pod \"multus-additional-cni-plugins-4cz4g\" (UID: \"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c\") " pod="openshift-multus/multus-additional-cni-plugins-4cz4g" Nov 03 21:44:45.141060 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.140285 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4cz4g\" (UID: \"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c\") " pod="openshift-multus/multus-additional-cni-plugins-4cz4g" Nov 03 21:44:45.141060 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.140314 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c-cnibin\") pod \"multus-additional-cni-plugins-4cz4g\" (UID: \"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c\") " pod="openshift-multus/multus-additional-cni-plugins-4cz4g" Nov 03 21:44:45.141060 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.140342 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-run-openvswitch\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.141060 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.140358 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-etc-modprobe-d\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.141060 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.140402 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg4r2\" (UniqueName: \"kubernetes.io/projected/9f5c183e-d58a-4621-940c-04a7ae0bafc9-kube-api-access-rg4r2\") pod \"node-resolver-9pctw\" (UID: \"9f5c183e-d58a-4621-940c-04a7ae0bafc9\") " pod="openshift-dns/node-resolver-9pctw" Nov 03 21:44:45.141060 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.140441 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-host-kubelet\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.141060 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.140540 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/219625cd-f1a4-4e51-90ee-58f0fc5939bb-multus-daemon-config\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.141060 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.140589 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgzcx\" (UniqueName: \"kubernetes.io/projected/eb5f720e-f465-439f-ba32-4f595561f9f6-kube-api-access-pgzcx\") pod \"network-metrics-daemon-knt57\" (UID: \"eb5f720e-f465-439f-ba32-4f595561f9f6\") " pod="openshift-multus/network-metrics-daemon-knt57" Nov 03 21:44:45.141060 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.140616 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/48d8b783-b602-47ef-9b9b-f2416d448908-sys-fs\") pod \"aws-ebs-csi-driver-node-wvd2c\" (UID: \"48d8b783-b602-47ef-9b9b-f2416d448908\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" Nov 03 21:44:45.141060 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.140635 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-run-ovn\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.141725 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.140651 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/76122905-15b4-4f4e-97b9-9f73a7675e36-env-overrides\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.141725 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.140677 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/76122905-15b4-4f4e-97b9-9f73a7675e36-ovnkube-script-lib\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.141725 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.140681 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4cz4g\" (UID: \"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c\") " pod="openshift-multus/multus-additional-cni-plugins-4cz4g" Nov 03 21:44:45.141725 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.140708 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/811514f5-baff-4fc1-9e92-28f27de7b0b7-konnectivity-ca\") pod \"konnectivity-agent-v98zn\" (UID: \"811514f5-baff-4fc1-9e92-28f27de7b0b7\") " pod="kube-system/konnectivity-agent-v98zn" Nov 03 21:44:45.141725 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.140733 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9c970a61-3ebf-48c9-a05b-688accd0b18a-etc-tuned\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.141725 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.140762 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9c970a61-3ebf-48c9-a05b-688accd0b18a-tmp\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.141725 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.140788 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f5c183e-d58a-4621-940c-04a7ae0bafc9-tmp-dir\") pod \"node-resolver-9pctw\" (UID: \"9f5c183e-d58a-4621-940c-04a7ae0bafc9\") " pod="openshift-dns/node-resolver-9pctw" Nov 03 21:44:45.141725 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.140832 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-host-cni-bin\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.141725 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.140909 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-etc-kubernetes\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.141725 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.140928 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-etc-kubernetes\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.141725 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.140943 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-etc-sysctl-d\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.141725 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.140965 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-run\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.141725 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.141080 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f5c183e-d58a-4621-940c-04a7ae0bafc9-tmp-dir\") pod \"node-resolver-9pctw\" (UID: \"9f5c183e-d58a-4621-940c-04a7ae0bafc9\") " pod="openshift-dns/node-resolver-9pctw" Nov 03 21:44:45.152405 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.152384 2578 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Nov 03 21:44:45.156101 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.155984 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-csdhx\" (UniqueName: \"kubernetes.io/projected/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c-kube-api-access-csdhx\") pod \"multus-additional-cni-plugins-4cz4g\" (UID: \"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c\") " pod="openshift-multus/multus-additional-cni-plugins-4cz4g" Nov 03 21:44:45.156228 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.155984 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg4r2\" (UniqueName: \"kubernetes.io/projected/9f5c183e-d58a-4621-940c-04a7ae0bafc9-kube-api-access-rg4r2\") pod \"node-resolver-9pctw\" (UID: \"9f5c183e-d58a-4621-940c-04a7ae0bafc9\") " pod="openshift-dns/node-resolver-9pctw" Nov 03 21:44:45.241606 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.241572 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0db6d946-a1d8-4459-a883-6f93f7e1182a-host-slash\") pod \"iptables-alerter-nkngm\" (UID: \"0db6d946-a1d8-4459-a883-6f93f7e1182a\") " pod="openshift-network-operator/iptables-alerter-nkngm" Nov 03 21:44:45.241606 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.241608 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/76122905-15b4-4f4e-97b9-9f73a7675e36-ovn-node-metrics-cert\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.241805 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.241626 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a440580e-b0bd-49e2-a33d-576118e049ae-serviceca\") pod \"node-ca-nz4zw\" (UID: \"a440580e-b0bd-49e2-a33d-576118e049ae\") " pod="openshift-image-registry/node-ca-nz4zw" Nov 03 21:44:45.241805 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.241680 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0db6d946-a1d8-4459-a883-6f93f7e1182a-host-slash\") pod \"iptables-alerter-nkngm\" (UID: \"0db6d946-a1d8-4459-a883-6f93f7e1182a\") " pod="openshift-network-operator/iptables-alerter-nkngm" Nov 03 21:44:45.241805 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.241718 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-var-lib-openvswitch\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.241805 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.241742 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/76122905-15b4-4f4e-97b9-9f73a7675e36-ovnkube-config\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.241805 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.241762 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a440580e-b0bd-49e2-a33d-576118e049ae-host\") pod \"node-ca-nz4zw\" (UID: \"a440580e-b0bd-49e2-a33d-576118e049ae\") " pod="openshift-image-registry/node-ca-nz4zw" Nov 03 21:44:45.241805 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.241783 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqplq\" (UniqueName: \"kubernetes.io/projected/a440580e-b0bd-49e2-a33d-576118e049ae-kube-api-access-lqplq\") pod \"node-ca-nz4zw\" (UID: \"a440580e-b0bd-49e2-a33d-576118e049ae\") " pod="openshift-image-registry/node-ca-nz4zw" Nov 03 21:44:45.241805 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.241805 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/48d8b783-b602-47ef-9b9b-f2416d448908-registration-dir\") pod \"aws-ebs-csi-driver-node-wvd2c\" (UID: \"48d8b783-b602-47ef-9b9b-f2416d448908\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" Nov 03 21:44:45.242116 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.241827 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-sys\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.242116 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.241844 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-var-lib-openvswitch\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.242116 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.241863 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a440580e-b0bd-49e2-a33d-576118e049ae-host\") pod \"node-ca-nz4zw\" (UID: \"a440580e-b0bd-49e2-a33d-576118e049ae\") " pod="openshift-image-registry/node-ca-nz4zw" Nov 03 21:44:45.242116 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.241853 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-host-slash\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.242116 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.241901 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/48d8b783-b602-47ef-9b9b-f2416d448908-registration-dir\") pod \"aws-ebs-csi-driver-node-wvd2c\" (UID: \"48d8b783-b602-47ef-9b9b-f2416d448908\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" Nov 03 21:44:45.242116 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.241901 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-host-slash\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.242116 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.241923 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-host-run-netns\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.242116 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.241948 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-sys\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.242116 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.241950 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-etc-openvswitch\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.242116 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.241977 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh4kw\" (UniqueName: \"kubernetes.io/projected/5d7e7316-1bd4-42cf-8519-544c1e2a4526-kube-api-access-zh4kw\") pod \"network-check-target-zrbsd\" (UID: \"5d7e7316-1bd4-42cf-8519-544c1e2a4526\") " pod="openshift-network-diagnostics/network-check-target-zrbsd" Nov 03 21:44:45.242116 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.241990 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-host-run-netns\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.242116 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.242003 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/94aaa3ac-c7fc-4f87-878e-8ef79127488b-original-pull-secret\") pod \"global-pull-secret-syncer-jq6hj\" (UID: \"94aaa3ac-c7fc-4f87-878e-8ef79127488b\") " pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:44:45.242116 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.242026 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-multus-cni-dir\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.242116 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.242060 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-host-run-netns\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.242116 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.242085 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-host-var-lib-cni-multus\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.242116 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.242108 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/48d8b783-b602-47ef-9b9b-f2416d448908-etc-selinux\") pod \"aws-ebs-csi-driver-node-wvd2c\" (UID: \"48d8b783-b602-47ef-9b9b-f2416d448908\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" Nov 03 21:44:45.242116 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.242144 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-host\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.242780 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.242160 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a440580e-b0bd-49e2-a33d-576118e049ae-serviceca\") pod \"node-ca-nz4zw\" (UID: \"a440580e-b0bd-49e2-a33d-576118e049ae\") " pod="openshift-image-registry/node-ca-nz4zw" Nov 03 21:44:45.242780 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:45.242163 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 03 21:44:45.242780 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.242165 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9k8xx\" (UniqueName: \"kubernetes.io/projected/9c970a61-3ebf-48c9-a05b-688accd0b18a-kube-api-access-9k8xx\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.242780 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.242196 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-host-run-netns\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.242780 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.242210 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-log-socket\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.242780 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.242026 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-etc-openvswitch\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.242780 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:45.242243 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94aaa3ac-c7fc-4f87-878e-8ef79127488b-original-pull-secret podName:94aaa3ac-c7fc-4f87-878e-8ef79127488b nodeName:}" failed. No retries permitted until 2026-11-03 21:44:45.742224709 +0000 UTC m=+2.051453719 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/94aaa3ac-c7fc-4f87-878e-8ef79127488b-original-pull-secret") pod "global-pull-secret-syncer-jq6hj" (UID: "94aaa3ac-c7fc-4f87-878e-8ef79127488b") : object "kube-system"/"original-pull-secret" not registered Nov 03 21:44:45.242780 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.242254 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-log-socket\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.242780 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.242273 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-run-openvswitch\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.242780 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.242301 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-run-openvswitch\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.242780 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.242308 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-multus-cni-dir\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.242780 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.242305 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-etc-modprobe-d\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.242780 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.242347 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-host-kubelet\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.242780 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.242372 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/219625cd-f1a4-4e51-90ee-58f0fc5939bb-multus-daemon-config\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.242780 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.242381 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/76122905-15b4-4f4e-97b9-9f73a7675e36-ovnkube-config\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.242780 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.242366 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-host-var-lib-cni-multus\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.242780 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.242393 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgzcx\" (UniqueName: \"kubernetes.io/projected/eb5f720e-f465-439f-ba32-4f595561f9f6-kube-api-access-pgzcx\") pod \"network-metrics-daemon-knt57\" (UID: \"eb5f720e-f465-439f-ba32-4f595561f9f6\") " pod="openshift-multus/network-metrics-daemon-knt57" Nov 03 21:44:45.243332 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.242415 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/48d8b783-b602-47ef-9b9b-f2416d448908-etc-selinux\") pod \"aws-ebs-csi-driver-node-wvd2c\" (UID: \"48d8b783-b602-47ef-9b9b-f2416d448908\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" Nov 03 21:44:45.243332 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.242430 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/48d8b783-b602-47ef-9b9b-f2416d448908-sys-fs\") pod \"aws-ebs-csi-driver-node-wvd2c\" (UID: \"48d8b783-b602-47ef-9b9b-f2416d448908\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" Nov 03 21:44:45.243332 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.242459 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-host-kubelet\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.243332 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.242466 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-run-ovn\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.243332 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.242461 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-etc-modprobe-d\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.243332 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.242522 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/48d8b783-b602-47ef-9b9b-f2416d448908-sys-fs\") pod \"aws-ebs-csi-driver-node-wvd2c\" (UID: \"48d8b783-b602-47ef-9b9b-f2416d448908\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" Nov 03 21:44:45.243332 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.242577 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/76122905-15b4-4f4e-97b9-9f73a7675e36-env-overrides\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.243332 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.242596 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-run-ovn\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.243332 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.242605 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/76122905-15b4-4f4e-97b9-9f73a7675e36-ovnkube-script-lib\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.243332 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.242654 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/811514f5-baff-4fc1-9e92-28f27de7b0b7-konnectivity-ca\") pod \"konnectivity-agent-v98zn\" (UID: \"811514f5-baff-4fc1-9e92-28f27de7b0b7\") " pod="kube-system/konnectivity-agent-v98zn" Nov 03 21:44:45.243332 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.242676 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-host\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.243332 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.242682 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9c970a61-3ebf-48c9-a05b-688accd0b18a-etc-tuned\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.243332 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.242723 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9c970a61-3ebf-48c9-a05b-688accd0b18a-tmp\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.243332 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.242773 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-host-cni-bin\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.243332 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.242806 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-etc-kubernetes\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.243332 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.242829 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-etc-kubernetes\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.243332 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.242852 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-etc-sysctl-d\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.243332 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.242968 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-etc-kubernetes\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.244096 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.242975 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/76122905-15b4-4f4e-97b9-9f73a7675e36-env-overrides\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.244096 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.242971 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/219625cd-f1a4-4e51-90ee-58f0fc5939bb-multus-daemon-config\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.244096 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243003 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-host-cni-bin\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.244096 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243036 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-run\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.244096 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243061 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0db6d946-a1d8-4459-a883-6f93f7e1182a-iptables-alerter-script\") pod \"iptables-alerter-nkngm\" (UID: \"0db6d946-a1d8-4459-a883-6f93f7e1182a\") " pod="openshift-network-operator/iptables-alerter-nkngm" Nov 03 21:44:45.244096 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243087 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-cnibin\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.244096 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243154 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-hostroot\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.244096 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243177 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/94aaa3ac-c7fc-4f87-878e-8ef79127488b-kubelet-config\") pod \"global-pull-secret-syncer-jq6hj\" (UID: \"94aaa3ac-c7fc-4f87-878e-8ef79127488b\") " pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:44:45.244096 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243198 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl5nr\" (UniqueName: \"kubernetes.io/projected/219625cd-f1a4-4e51-90ee-58f0fc5939bb-kube-api-access-kl5nr\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.244096 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243223 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/48d8b783-b602-47ef-9b9b-f2416d448908-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wvd2c\" (UID: \"48d8b783-b602-47ef-9b9b-f2416d448908\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" Nov 03 21:44:45.244096 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243261 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/811514f5-baff-4fc1-9e92-28f27de7b0b7-konnectivity-ca\") pod \"konnectivity-agent-v98zn\" (UID: \"811514f5-baff-4fc1-9e92-28f27de7b0b7\") " pod="kube-system/konnectivity-agent-v98zn" Nov 03 21:44:45.244096 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243282 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-host-var-lib-cni-bin\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.244096 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243305 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/48d8b783-b602-47ef-9b9b-f2416d448908-socket-dir\") pod \"aws-ebs-csi-driver-node-wvd2c\" (UID: \"48d8b783-b602-47ef-9b9b-f2416d448908\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" Nov 03 21:44:45.244096 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243328 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/48d8b783-b602-47ef-9b9b-f2416d448908-device-dir\") pod \"aws-ebs-csi-driver-node-wvd2c\" (UID: \"48d8b783-b602-47ef-9b9b-f2416d448908\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" Nov 03 21:44:45.244096 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243354 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-etc-sysctl-conf\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.244096 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243367 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-etc-kubernetes\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.244096 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243379 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-etc-systemd\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.244096 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243325 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/76122905-15b4-4f4e-97b9-9f73a7675e36-ovnkube-script-lib\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.244954 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243381 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/48d8b783-b602-47ef-9b9b-f2416d448908-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wvd2c\" (UID: \"48d8b783-b602-47ef-9b9b-f2416d448908\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" Nov 03 21:44:45.244954 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243413 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-hostroot\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.244954 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243427 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-host-cni-netd\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.244954 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243435 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-cnibin\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.244954 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243434 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-host-var-lib-cni-bin\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.244954 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243454 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.244954 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243479 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/94aaa3ac-c7fc-4f87-878e-8ef79127488b-dbus\") pod \"global-pull-secret-syncer-jq6hj\" (UID: \"94aaa3ac-c7fc-4f87-878e-8ef79127488b\") " pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:44:45.244954 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243481 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-etc-sysctl-d\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.244954 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243508 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/48d8b783-b602-47ef-9b9b-f2416d448908-socket-dir\") pod \"aws-ebs-csi-driver-node-wvd2c\" (UID: \"48d8b783-b602-47ef-9b9b-f2416d448908\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" Nov 03 21:44:45.244954 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243531 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/94aaa3ac-c7fc-4f87-878e-8ef79127488b-dbus\") pod \"global-pull-secret-syncer-jq6hj\" (UID: \"94aaa3ac-c7fc-4f87-878e-8ef79127488b\") " pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:44:45.244954 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243555 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-etc-sysctl-conf\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.244954 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243573 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/94aaa3ac-c7fc-4f87-878e-8ef79127488b-kubelet-config\") pod \"global-pull-secret-syncer-jq6hj\" (UID: \"94aaa3ac-c7fc-4f87-878e-8ef79127488b\") " pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:44:45.244954 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243574 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-host-cni-netd\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.244954 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243513 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6c6w8\" (UniqueName: \"kubernetes.io/projected/0db6d946-a1d8-4459-a883-6f93f7e1182a-kube-api-access-6c6w8\") pod \"iptables-alerter-nkngm\" (UID: \"0db6d946-a1d8-4459-a883-6f93f7e1182a\") " pod="openshift-network-operator/iptables-alerter-nkngm" Nov 03 21:44:45.244954 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243604 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-etc-systemd\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.244954 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243651 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.244954 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243686 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-host-run-k8s-cni-cncf-io\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.245800 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243694 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/48d8b783-b602-47ef-9b9b-f2416d448908-device-dir\") pod \"aws-ebs-csi-driver-node-wvd2c\" (UID: \"48d8b783-b602-47ef-9b9b-f2416d448908\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" Nov 03 21:44:45.245800 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243691 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-run\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.245800 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243735 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-systemd-units\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.245800 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243737 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-host-run-k8s-cni-cncf-io\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.245800 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243776 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-run-systemd\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.245800 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243811 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-systemd-units\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.245800 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243820 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmr4b\" (UniqueName: \"kubernetes.io/projected/76122905-15b4-4f4e-97b9-9f73a7675e36-kube-api-access-tmr4b\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.245800 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243847 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-run-systemd\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.245800 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243895 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-host-run-ovn-kubernetes\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.245800 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243937 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-host-run-ovn-kubernetes\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.245800 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243938 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/811514f5-baff-4fc1-9e92-28f27de7b0b7-agent-certs\") pod \"konnectivity-agent-v98zn\" (UID: \"811514f5-baff-4fc1-9e92-28f27de7b0b7\") " pod="kube-system/konnectivity-agent-v98zn" Nov 03 21:44:45.245800 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.243972 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-multus-conf-dir\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.245800 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.244057 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-multus-conf-dir\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.245800 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.244464 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/76122905-15b4-4f4e-97b9-9f73a7675e36-ovn-node-metrics-cert\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.245800 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.244480 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0db6d946-a1d8-4459-a883-6f93f7e1182a-iptables-alerter-script\") pod \"iptables-alerter-nkngm\" (UID: \"0db6d946-a1d8-4459-a883-6f93f7e1182a\") " pod="openshift-network-operator/iptables-alerter-nkngm" Nov 03 21:44:45.245800 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.245358 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9c970a61-3ebf-48c9-a05b-688accd0b18a-etc-tuned\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.245800 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.245649 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb5f720e-f465-439f-ba32-4f595561f9f6-metrics-certs\") pod \"network-metrics-daemon-knt57\" (UID: \"eb5f720e-f465-439f-ba32-4f595561f9f6\") " pod="openshift-multus/network-metrics-daemon-knt57" Nov 03 21:44:45.246599 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.245684 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-var-lib-kubelet\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.246599 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.245711 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-multus-socket-dir-parent\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.246599 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.245717 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9c970a61-3ebf-48c9-a05b-688accd0b18a-tmp\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.246599 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:45.245770 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 03 21:44:45.246599 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.245766 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-var-lib-kubelet\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.246599 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.245781 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-multus-socket-dir-parent\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.246599 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.245811 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-host-run-multus-certs\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.246599 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:45.245837 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb5f720e-f465-439f-ba32-4f595561f9f6-metrics-certs podName:eb5f720e-f465-439f-ba32-4f595561f9f6 nodeName:}" failed. No retries permitted until 2026-11-03 21:44:45.745824885 +0000 UTC m=+2.055053882 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eb5f720e-f465-439f-ba32-4f595561f9f6-metrics-certs") pod "network-metrics-daemon-knt57" (UID: "eb5f720e-f465-439f-ba32-4f595561f9f6") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 03 21:44:45.246599 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.245847 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-host-run-multus-certs\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.246599 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.245852 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-node-log\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.246599 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.245877 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/219625cd-f1a4-4e51-90ee-58f0fc5939bb-cni-binary-copy\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.246599 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.245892 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-host-var-lib-kubelet\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.246599 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.245908 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8fqs\" (UniqueName: \"kubernetes.io/projected/48d8b783-b602-47ef-9b9b-f2416d448908-kube-api-access-k8fqs\") pod \"aws-ebs-csi-driver-node-wvd2c\" (UID: \"48d8b783-b602-47ef-9b9b-f2416d448908\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" Nov 03 21:44:45.246599 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.245911 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-node-log\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.246599 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.245923 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-etc-sysconfig\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.246599 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.245942 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-system-cni-dir\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.246599 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.245949 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-host-var-lib-kubelet\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.247393 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.245965 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-lib-modules\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.247393 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.245985 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-etc-sysconfig\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.247393 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.246009 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-system-cni-dir\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.247393 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.246054 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-lib-modules\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.247393 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.246330 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/219625cd-f1a4-4e51-90ee-58f0fc5939bb-cni-binary-copy\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.247393 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.246394 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/811514f5-baff-4fc1-9e92-28f27de7b0b7-agent-certs\") pod \"konnectivity-agent-v98zn\" (UID: \"811514f5-baff-4fc1-9e92-28f27de7b0b7\") " pod="kube-system/konnectivity-agent-v98zn" Nov 03 21:44:45.258181 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:45.258109 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 03 21:44:45.258181 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:45.258143 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 03 21:44:45.258181 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:45.258157 2578 projected.go:196] Error preparing data for projected volume kube-api-access-zh4kw for pod openshift-network-diagnostics/network-check-target-zrbsd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 03 21:44:45.258865 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:45.258240 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5d7e7316-1bd4-42cf-8519-544c1e2a4526-kube-api-access-zh4kw podName:5d7e7316-1bd4-42cf-8519-544c1e2a4526 nodeName:}" failed. No retries permitted until 2026-11-03 21:44:45.758222855 +0000 UTC m=+2.067451880 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zh4kw" (UniqueName: "kubernetes.io/projected/5d7e7316-1bd4-42cf-8519-544c1e2a4526-kube-api-access-zh4kw") pod "network-check-target-zrbsd" (UID: "5d7e7316-1bd4-42cf-8519-544c1e2a4526") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 03 21:44:45.259307 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.259264 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqplq\" (UniqueName: \"kubernetes.io/projected/a440580e-b0bd-49e2-a33d-576118e049ae-kube-api-access-lqplq\") pod \"node-ca-nz4zw\" (UID: \"a440580e-b0bd-49e2-a33d-576118e049ae\") " pod="openshift-image-registry/node-ca-nz4zw" Nov 03 21:44:45.260418 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.260391 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8fqs\" (UniqueName: \"kubernetes.io/projected/48d8b783-b602-47ef-9b9b-f2416d448908-kube-api-access-k8fqs\") pod \"aws-ebs-csi-driver-node-wvd2c\" (UID: \"48d8b783-b602-47ef-9b9b-f2416d448908\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" Nov 03 21:44:45.260723 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.260703 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl5nr\" (UniqueName: \"kubernetes.io/projected/219625cd-f1a4-4e51-90ee-58f0fc5939bb-kube-api-access-kl5nr\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.260787 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.260731 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9k8xx\" (UniqueName: \"kubernetes.io/projected/9c970a61-3ebf-48c9-a05b-688accd0b18a-kube-api-access-9k8xx\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.260827 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.260789 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmr4b\" (UniqueName: \"kubernetes.io/projected/76122905-15b4-4f4e-97b9-9f73a7675e36-kube-api-access-tmr4b\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.260943 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.260929 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6c6w8\" (UniqueName: \"kubernetes.io/projected/0db6d946-a1d8-4459-a883-6f93f7e1182a-kube-api-access-6c6w8\") pod \"iptables-alerter-nkngm\" (UID: \"0db6d946-a1d8-4459-a883-6f93f7e1182a\") " pod="openshift-network-operator/iptables-alerter-nkngm" Nov 03 21:44:45.261544 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.261526 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgzcx\" (UniqueName: \"kubernetes.io/projected/eb5f720e-f465-439f-ba32-4f595561f9f6-kube-api-access-pgzcx\") pod \"network-metrics-daemon-knt57\" (UID: \"eb5f720e-f465-439f-ba32-4f595561f9f6\") " pod="openshift-multus/network-metrics-daemon-knt57" Nov 03 21:44:45.360714 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.360628 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9pctw" Nov 03 21:44:45.379433 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.379406 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4cz4g" Nov 03 21:44:45.386410 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.386383 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mbksx" Nov 03 21:44:45.405342 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.405316 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:44:45.417227 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.417204 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" Nov 03 21:44:45.427998 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.427974 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:44:45.434589 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.434563 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-v98zn" Nov 03 21:44:45.453383 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.453351 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nz4zw" Nov 03 21:44:45.458074 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.458050 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nkngm" Nov 03 21:44:45.492848 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:45.492818 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58ba030777ca53e38084de26650a107e.slice/crio-cae35dc92e36aaa91e303dd7f694cff5e2cc1a2b8830c0eda5a4496a101f574e WatchSource:0}: Error finding container cae35dc92e36aaa91e303dd7f694cff5e2cc1a2b8830c0eda5a4496a101f574e: Status 404 returned error can't find the container with id cae35dc92e36aaa91e303dd7f694cff5e2cc1a2b8830c0eda5a4496a101f574e Nov 03 21:44:45.493295 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:45.493237 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70b8abbfbbbc47301b7da80ca6b44429.slice/crio-c244c1362b3ac28277d760dd45ee267608cffbbcd8811e1b3de0d2f5e23796df WatchSource:0}: Error finding container c244c1362b3ac28277d760dd45ee267608cffbbcd8811e1b3de0d2f5e23796df: Status 404 returned error can't find the container with id c244c1362b3ac28277d760dd45ee267608cffbbcd8811e1b3de0d2f5e23796df Nov 03 21:44:45.493892 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:45.493872 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0db6d946_a1d8_4459_a883_6f93f7e1182a.slice/crio-93cbd093a41d6f8280f48bbb0d4906b6cf9bcedd415031f1e155400be7c35b12 WatchSource:0}: Error finding container 93cbd093a41d6f8280f48bbb0d4906b6cf9bcedd415031f1e155400be7c35b12: Status 404 returned error can't find the container with id 93cbd093a41d6f8280f48bbb0d4906b6cf9bcedd415031f1e155400be7c35b12 Nov 03 21:44:45.498785 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.498768 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 03 21:44:45.631229 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.631149 2578 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Nov 03 21:44:45.727906 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:45.727877 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f5c183e_d58a_4621_940c_04a7ae0bafc9.slice/crio-3f1b9e13ddbf6ca5a0b73da16e9f34643e98cbe21681f3aa9c6c28106302b181 WatchSource:0}: Error finding container 3f1b9e13ddbf6ca5a0b73da16e9f34643e98cbe21681f3aa9c6c28106302b181: Status 404 returned error can't find the container with id 3f1b9e13ddbf6ca5a0b73da16e9f34643e98cbe21681f3aa9c6c28106302b181 Nov 03 21:44:45.749629 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.749598 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb5f720e-f465-439f-ba32-4f595561f9f6-metrics-certs\") pod \"network-metrics-daemon-knt57\" (UID: \"eb5f720e-f465-439f-ba32-4f595561f9f6\") " pod="openshift-multus/network-metrics-daemon-knt57" Nov 03 21:44:45.749756 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.749648 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/94aaa3ac-c7fc-4f87-878e-8ef79127488b-original-pull-secret\") pod \"global-pull-secret-syncer-jq6hj\" (UID: \"94aaa3ac-c7fc-4f87-878e-8ef79127488b\") " pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:44:45.749756 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:45.749748 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 03 21:44:45.749828 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:45.749751 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 03 21:44:45.749828 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:45.749796 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94aaa3ac-c7fc-4f87-878e-8ef79127488b-original-pull-secret podName:94aaa3ac-c7fc-4f87-878e-8ef79127488b nodeName:}" failed. No retries permitted until 2026-11-03 21:44:46.749783501 +0000 UTC m=+3.059012494 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/94aaa3ac-c7fc-4f87-878e-8ef79127488b-original-pull-secret") pod "global-pull-secret-syncer-jq6hj" (UID: "94aaa3ac-c7fc-4f87-878e-8ef79127488b") : object "kube-system"/"original-pull-secret" not registered Nov 03 21:44:45.749828 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:45.749808 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb5f720e-f465-439f-ba32-4f595561f9f6-metrics-certs podName:eb5f720e-f465-439f-ba32-4f595561f9f6 nodeName:}" failed. No retries permitted until 2026-11-03 21:44:46.749802184 +0000 UTC m=+3.059031177 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eb5f720e-f465-439f-ba32-4f595561f9f6-metrics-certs") pod "network-metrics-daemon-knt57" (UID: "eb5f720e-f465-439f-ba32-4f595561f9f6") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 03 21:44:45.771531 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:45.771500 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62e082c7_e3d4_4b77_a2ea_b8f41fc9e41c.slice/crio-3bb48199a050488f952dbd7eefe0990c61e8638c924e084fbe0e9105710f7be6 WatchSource:0}: Error finding container 3bb48199a050488f952dbd7eefe0990c61e8638c924e084fbe0e9105710f7be6: Status 404 returned error can't find the container with id 3bb48199a050488f952dbd7eefe0990c61e8638c924e084fbe0e9105710f7be6 Nov 03 21:44:45.772035 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:45.772013 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod219625cd_f1a4_4e51_90ee_58f0fc5939bb.slice/crio-14b5af6921fb45781a99bda05244046395650b5904b97226a33bfda38089f77f WatchSource:0}: Error finding container 14b5af6921fb45781a99bda05244046395650b5904b97226a33bfda38089f77f: Status 404 returned error can't find the container with id 14b5af6921fb45781a99bda05244046395650b5904b97226a33bfda38089f77f Nov 03 21:44:45.828999 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:45.828959 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76122905_15b4_4f4e_97b9_9f73a7675e36.slice/crio-484d94934d6014e3bd2e5fcfb68db6488c20d2d1ec8cefbc9075eeab26519f20 WatchSource:0}: Error finding container 484d94934d6014e3bd2e5fcfb68db6488c20d2d1ec8cefbc9075eeab26519f20: Status 404 returned error can't find the container with id 484d94934d6014e3bd2e5fcfb68db6488c20d2d1ec8cefbc9075eeab26519f20 Nov 03 21:44:45.829620 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:45.829596 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c970a61_3ebf_48c9_a05b_688accd0b18a.slice/crio-b43ddb07b88c30829a1db4ab2e9aa1777a940796617a25b686b3497d25c745c7 WatchSource:0}: Error finding container b43ddb07b88c30829a1db4ab2e9aa1777a940796617a25b686b3497d25c745c7: Status 404 returned error can't find the container with id b43ddb07b88c30829a1db4ab2e9aa1777a940796617a25b686b3497d25c745c7 Nov 03 21:44:45.850991 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:45.850960 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh4kw\" (UniqueName: \"kubernetes.io/projected/5d7e7316-1bd4-42cf-8519-544c1e2a4526-kube-api-access-zh4kw\") pod \"network-check-target-zrbsd\" (UID: \"5d7e7316-1bd4-42cf-8519-544c1e2a4526\") " pod="openshift-network-diagnostics/network-check-target-zrbsd" Nov 03 21:44:45.851209 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:45.851114 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 03 21:44:45.851258 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:45.851214 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 03 21:44:45.851258 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:45.851225 2578 projected.go:196] Error preparing data for projected volume kube-api-access-zh4kw for pod openshift-network-diagnostics/network-check-target-zrbsd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 03 21:44:45.851318 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:45.851269 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5d7e7316-1bd4-42cf-8519-544c1e2a4526-kube-api-access-zh4kw podName:5d7e7316-1bd4-42cf-8519-544c1e2a4526 nodeName:}" failed. No retries permitted until 2026-11-03 21:44:46.851256483 +0000 UTC m=+3.160485476 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-zh4kw" (UniqueName: "kubernetes.io/projected/5d7e7316-1bd4-42cf-8519-544c1e2a4526-kube-api-access-zh4kw") pod "network-check-target-zrbsd" (UID: "5d7e7316-1bd4-42cf-8519-544c1e2a4526") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 03 21:44:45.855257 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:45.855217 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod811514f5_baff_4fc1_9e92_28f27de7b0b7.slice/crio-8783df8686233a4ddb7fe45426c063a89d5e82fbd6273021e42a43027efbd5de WatchSource:0}: Error finding container 8783df8686233a4ddb7fe45426c063a89d5e82fbd6273021e42a43027efbd5de: Status 404 returned error can't find the container with id 8783df8686233a4ddb7fe45426c063a89d5e82fbd6273021e42a43027efbd5de Nov 03 21:44:45.871503 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:45.871477 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48d8b783_b602_47ef_9b9b_f2416d448908.slice/crio-da3b1b4886307f48cd0e54b04214c86fafa13ec3b4d3891c843639464ccd9707 WatchSource:0}: Error finding container da3b1b4886307f48cd0e54b04214c86fafa13ec3b4d3891c843639464ccd9707: Status 404 returned error can't find the container with id da3b1b4886307f48cd0e54b04214c86fafa13ec3b4d3891c843639464ccd9707 Nov 03 21:44:45.892421 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:44:45.892396 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda440580e_b0bd_49e2_a33d_576118e049ae.slice/crio-1b370a6c830e98f01be8c325b5aecac0f16f4ab08328435306f38e3b5b58aea7 WatchSource:0}: Error finding container 1b370a6c830e98f01be8c325b5aecac0f16f4ab08328435306f38e3b5b58aea7: Status 404 returned error can't find the container with id 1b370a6c830e98f01be8c325b5aecac0f16f4ab08328435306f38e3b5b58aea7 Nov 03 21:44:46.082764 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:46.082693 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-02 21:39:45 +0000 UTC" deadline="2028-08-19 05:56:01.562801232 +0000 UTC" Nov 03 21:44:46.082764 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:46.082764 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15704h11m15.48004236s" Nov 03 21:44:46.373197 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:46.371832 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrbsd" Nov 03 21:44:46.373197 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:46.371969 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrbsd" podUID="5d7e7316-1bd4-42cf-8519-544c1e2a4526" Nov 03 21:44:46.373197 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:46.372424 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:44:46.373197 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:46.372523 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jq6hj" podUID="94aaa3ac-c7fc-4f87-878e-8ef79127488b" Nov 03 21:44:46.384421 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:46.384321 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nkngm" event={"ID":"0db6d946-a1d8-4459-a883-6f93f7e1182a","Type":"ContainerStarted","Data":"93cbd093a41d6f8280f48bbb0d4906b6cf9bcedd415031f1e155400be7c35b12"} Nov 03 21:44:46.400801 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:46.400762 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-7.us-east-2.compute.internal" event={"ID":"70b8abbfbbbc47301b7da80ca6b44429","Type":"ContainerStarted","Data":"c244c1362b3ac28277d760dd45ee267608cffbbcd8811e1b3de0d2f5e23796df"} Nov 03 21:44:46.411065 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:46.411035 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nz4zw" event={"ID":"a440580e-b0bd-49e2-a33d-576118e049ae","Type":"ContainerStarted","Data":"1b370a6c830e98f01be8c325b5aecac0f16f4ab08328435306f38e3b5b58aea7"} Nov 03 21:44:46.421045 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:46.420897 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" event={"ID":"48d8b783-b602-47ef-9b9b-f2416d448908","Type":"ContainerStarted","Data":"da3b1b4886307f48cd0e54b04214c86fafa13ec3b4d3891c843639464ccd9707"} Nov 03 21:44:46.425304 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:46.424242 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" event={"ID":"9c970a61-3ebf-48c9-a05b-688accd0b18a","Type":"ContainerStarted","Data":"b43ddb07b88c30829a1db4ab2e9aa1777a940796617a25b686b3497d25c745c7"} Nov 03 21:44:46.431228 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:46.431192 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mbksx" event={"ID":"219625cd-f1a4-4e51-90ee-58f0fc5939bb","Type":"ContainerStarted","Data":"14b5af6921fb45781a99bda05244046395650b5904b97226a33bfda38089f77f"} Nov 03 21:44:46.446195 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:46.445998 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4cz4g" event={"ID":"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c","Type":"ContainerStarted","Data":"3bb48199a050488f952dbd7eefe0990c61e8638c924e084fbe0e9105710f7be6"} Nov 03 21:44:46.454731 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:46.454699 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-v98zn" event={"ID":"811514f5-baff-4fc1-9e92-28f27de7b0b7","Type":"ContainerStarted","Data":"8783df8686233a4ddb7fe45426c063a89d5e82fbd6273021e42a43027efbd5de"} Nov 03 21:44:46.480236 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:46.480205 2578 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Nov 03 21:44:46.486883 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:46.486715 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" event={"ID":"76122905-15b4-4f4e-97b9-9f73a7675e36","Type":"ContainerStarted","Data":"484d94934d6014e3bd2e5fcfb68db6488c20d2d1ec8cefbc9075eeab26519f20"} Nov 03 21:44:46.505065 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:46.504999 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9pctw" event={"ID":"9f5c183e-d58a-4621-940c-04a7ae0bafc9","Type":"ContainerStarted","Data":"3f1b9e13ddbf6ca5a0b73da16e9f34643e98cbe21681f3aa9c6c28106302b181"} Nov 03 21:44:46.519141 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:46.519092 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal" event={"ID":"58ba030777ca53e38084de26650a107e","Type":"ContainerStarted","Data":"cae35dc92e36aaa91e303dd7f694cff5e2cc1a2b8830c0eda5a4496a101f574e"} Nov 03 21:44:46.758372 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:46.758281 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb5f720e-f465-439f-ba32-4f595561f9f6-metrics-certs\") pod \"network-metrics-daemon-knt57\" (UID: \"eb5f720e-f465-439f-ba32-4f595561f9f6\") " pod="openshift-multus/network-metrics-daemon-knt57" Nov 03 21:44:46.758372 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:46.758358 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/94aaa3ac-c7fc-4f87-878e-8ef79127488b-original-pull-secret\") pod \"global-pull-secret-syncer-jq6hj\" (UID: \"94aaa3ac-c7fc-4f87-878e-8ef79127488b\") " pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:44:46.758545 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:46.758508 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 03 21:44:46.758578 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:46.758566 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94aaa3ac-c7fc-4f87-878e-8ef79127488b-original-pull-secret podName:94aaa3ac-c7fc-4f87-878e-8ef79127488b nodeName:}" failed. No retries permitted until 2026-11-03 21:44:48.758548959 +0000 UTC m=+5.067777966 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/94aaa3ac-c7fc-4f87-878e-8ef79127488b-original-pull-secret") pod "global-pull-secret-syncer-jq6hj" (UID: "94aaa3ac-c7fc-4f87-878e-8ef79127488b") : object "kube-system"/"original-pull-secret" not registered Nov 03 21:44:46.758971 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:46.758952 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 03 21:44:46.759039 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:46.759005 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb5f720e-f465-439f-ba32-4f595561f9f6-metrics-certs podName:eb5f720e-f465-439f-ba32-4f595561f9f6 nodeName:}" failed. No retries permitted until 2026-11-03 21:44:48.758990705 +0000 UTC m=+5.068219711 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eb5f720e-f465-439f-ba32-4f595561f9f6-metrics-certs") pod "network-metrics-daemon-knt57" (UID: "eb5f720e-f465-439f-ba32-4f595561f9f6") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 03 21:44:46.859244 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:46.859210 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh4kw\" (UniqueName: \"kubernetes.io/projected/5d7e7316-1bd4-42cf-8519-544c1e2a4526-kube-api-access-zh4kw\") pod \"network-check-target-zrbsd\" (UID: \"5d7e7316-1bd4-42cf-8519-544c1e2a4526\") " pod="openshift-network-diagnostics/network-check-target-zrbsd" Nov 03 21:44:46.859430 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:46.859413 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 03 21:44:46.859494 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:46.859438 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 03 21:44:46.859494 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:46.859450 2578 projected.go:196] Error preparing data for projected volume kube-api-access-zh4kw for pod openshift-network-diagnostics/network-check-target-zrbsd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 03 21:44:46.859588 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:46.859507 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5d7e7316-1bd4-42cf-8519-544c1e2a4526-kube-api-access-zh4kw podName:5d7e7316-1bd4-42cf-8519-544c1e2a4526 nodeName:}" failed. No retries permitted until 2026-11-03 21:44:48.85948845 +0000 UTC m=+5.168717446 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-zh4kw" (UniqueName: "kubernetes.io/projected/5d7e7316-1bd4-42cf-8519-544c1e2a4526-kube-api-access-zh4kw") pod "network-check-target-zrbsd" (UID: "5d7e7316-1bd4-42cf-8519-544c1e2a4526") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 03 21:44:47.083504 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:47.083396 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-02 21:39:45 +0000 UTC" deadline="2028-06-07 04:53:50.094779167 +0000 UTC" Nov 03 21:44:47.083504 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:47.083463 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13951h9m3.011320925s" Nov 03 21:44:47.372253 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:47.372167 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knt57" Nov 03 21:44:47.372425 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:47.372345 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-knt57" podUID="eb5f720e-f465-439f-ba32-4f595561f9f6" Nov 03 21:44:48.000017 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:47.999987 2578 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Nov 03 21:44:48.115572 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:48.115487 2578 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Nov 03 21:44:48.378210 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:48.377864 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:44:48.378210 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:48.377981 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jq6hj" podUID="94aaa3ac-c7fc-4f87-878e-8ef79127488b" Nov 03 21:44:48.378428 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:48.378374 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrbsd" Nov 03 21:44:48.378702 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:48.378474 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrbsd" podUID="5d7e7316-1bd4-42cf-8519-544c1e2a4526" Nov 03 21:44:48.778733 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:48.778693 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/94aaa3ac-c7fc-4f87-878e-8ef79127488b-original-pull-secret\") pod \"global-pull-secret-syncer-jq6hj\" (UID: \"94aaa3ac-c7fc-4f87-878e-8ef79127488b\") " pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:44:48.778973 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:48.778770 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb5f720e-f465-439f-ba32-4f595561f9f6-metrics-certs\") pod \"network-metrics-daemon-knt57\" (UID: \"eb5f720e-f465-439f-ba32-4f595561f9f6\") " pod="openshift-multus/network-metrics-daemon-knt57" Nov 03 21:44:48.778973 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:48.778905 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 03 21:44:48.778973 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:48.778965 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb5f720e-f465-439f-ba32-4f595561f9f6-metrics-certs podName:eb5f720e-f465-439f-ba32-4f595561f9f6 nodeName:}" failed. No retries permitted until 2026-11-03 21:44:52.778947757 +0000 UTC m=+9.088176753 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eb5f720e-f465-439f-ba32-4f595561f9f6-metrics-certs") pod "network-metrics-daemon-knt57" (UID: "eb5f720e-f465-439f-ba32-4f595561f9f6") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 03 21:44:48.779418 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:48.779399 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 03 21:44:48.779490 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:48.779455 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94aaa3ac-c7fc-4f87-878e-8ef79127488b-original-pull-secret podName:94aaa3ac-c7fc-4f87-878e-8ef79127488b nodeName:}" failed. No retries permitted until 2026-11-03 21:44:52.779440289 +0000 UTC m=+9.088669297 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/94aaa3ac-c7fc-4f87-878e-8ef79127488b-original-pull-secret") pod "global-pull-secret-syncer-jq6hj" (UID: "94aaa3ac-c7fc-4f87-878e-8ef79127488b") : object "kube-system"/"original-pull-secret" not registered Nov 03 21:44:48.880729 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:48.880040 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh4kw\" (UniqueName: \"kubernetes.io/projected/5d7e7316-1bd4-42cf-8519-544c1e2a4526-kube-api-access-zh4kw\") pod \"network-check-target-zrbsd\" (UID: \"5d7e7316-1bd4-42cf-8519-544c1e2a4526\") " pod="openshift-network-diagnostics/network-check-target-zrbsd" Nov 03 21:44:48.880729 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:48.880274 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 03 21:44:48.880729 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:48.880292 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 03 21:44:48.880729 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:48.880306 2578 projected.go:196] Error preparing data for projected volume kube-api-access-zh4kw for pod openshift-network-diagnostics/network-check-target-zrbsd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 03 21:44:48.880729 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:48.880367 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5d7e7316-1bd4-42cf-8519-544c1e2a4526-kube-api-access-zh4kw podName:5d7e7316-1bd4-42cf-8519-544c1e2a4526 nodeName:}" failed. No retries permitted until 2026-11-03 21:44:52.880348646 +0000 UTC m=+9.189577643 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-zh4kw" (UniqueName: "kubernetes.io/projected/5d7e7316-1bd4-42cf-8519-544c1e2a4526-kube-api-access-zh4kw") pod "network-check-target-zrbsd" (UID: "5d7e7316-1bd4-42cf-8519-544c1e2a4526") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 03 21:44:49.372369 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:49.372330 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knt57" Nov 03 21:44:49.373070 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:49.372482 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-knt57" podUID="eb5f720e-f465-439f-ba32-4f595561f9f6" Nov 03 21:44:49.549298 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:49.548952 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4cz4g" event={"ID":"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c","Type":"ContainerStarted","Data":"32e5e852a07ab99b25cf2390a0d6f5516d694632654f82058ab02ed363be305b"} Nov 03 21:44:49.557835 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:49.557168 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal" event={"ID":"58ba030777ca53e38084de26650a107e","Type":"ContainerStarted","Data":"cfde88035187ebb9aabafdac7ec45b9549ab3ac9180a83951ebcf5b331611c03"} Nov 03 21:44:50.377149 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:50.376783 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:44:50.377149 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:50.376903 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jq6hj" podUID="94aaa3ac-c7fc-4f87-878e-8ef79127488b" Nov 03 21:44:50.377149 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:50.376966 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrbsd" Nov 03 21:44:50.377149 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:50.377033 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrbsd" podUID="5d7e7316-1bd4-42cf-8519-544c1e2a4526" Nov 03 21:44:50.565345 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:50.564507 2578 generic.go:356] "Generic (PLEG): container finished" podID="58ba030777ca53e38084de26650a107e" containerID="cfde88035187ebb9aabafdac7ec45b9549ab3ac9180a83951ebcf5b331611c03" exitCode=0 Nov 03 21:44:50.565345 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:50.564593 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal" event={"ID":"58ba030777ca53e38084de26650a107e","Type":"ContainerDied","Data":"cfde88035187ebb9aabafdac7ec45b9549ab3ac9180a83951ebcf5b331611c03"} Nov 03 21:44:50.575107 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:50.573493 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-7.us-east-2.compute.internal" event={"ID":"70b8abbfbbbc47301b7da80ca6b44429","Type":"ContainerStarted","Data":"78d9df0a1516cc2535f73127c8f004dcb94dbd2eafea2a5bb9d643361b6d2428"} Nov 03 21:44:50.578046 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:50.577399 2578 generic.go:356] "Generic (PLEG): container finished" podID="62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c" containerID="32e5e852a07ab99b25cf2390a0d6f5516d694632654f82058ab02ed363be305b" exitCode=0 Nov 03 21:44:50.578046 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:50.577480 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4cz4g" event={"ID":"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c","Type":"ContainerDied","Data":"32e5e852a07ab99b25cf2390a0d6f5516d694632654f82058ab02ed363be305b"} Nov 03 21:44:50.581990 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:50.581543 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-v98zn" event={"ID":"811514f5-baff-4fc1-9e92-28f27de7b0b7","Type":"ContainerStarted","Data":"fa50afc43dde2c88f9be2a3db07a02f7d0049f5e407e985d5d206d55361d232e"} Nov 03 21:44:50.630608 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:50.630546 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-v98zn" podStartSLOduration=2.351962189 podStartE2EDuration="6.630527025s" totalImagesPullingTime="4.278564836s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:44:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:44:50.629759566 +0000 UTC m=+6.938988594" watchObservedRunningTime="2026-11-03 21:44:50.630527025 +0000 UTC m=+6.939756045" Nov 03 21:44:51.371955 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:51.371915 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knt57" Nov 03 21:44:51.372114 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:51.372081 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-knt57" podUID="eb5f720e-f465-439f-ba32-4f595561f9f6" Nov 03 21:44:51.387604 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:51.387570 2578 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-v98zn" Nov 03 21:44:51.390269 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:51.390245 2578 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-v98zn" Nov 03 21:44:51.408627 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:51.407544 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-7.us-east-2.compute.internal" podStartSLOduration=7.40752726 podStartE2EDuration="7.40752726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 21:44:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:44:50.646983815 +0000 UTC m=+6.956212832" watchObservedRunningTime="2026-11-03 21:44:51.40752726 +0000 UTC m=+7.716756274" Nov 03 21:44:51.587916 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:51.587834 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal" event={"ID":"58ba030777ca53e38084de26650a107e","Type":"ContainerStarted","Data":"b48be4756b3a13ddbd10852a623be839e5ef4bd3482585882307a1970c21e124"} Nov 03 21:44:51.593485 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:51.593454 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nz4zw" event={"ID":"a440580e-b0bd-49e2-a33d-576118e049ae","Type":"ContainerStarted","Data":"999ae0c11d2b155311e0eedce4b529daf3f07e11488092b089a9145c36b5f75a"} Nov 03 21:44:51.624617 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:51.624555 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal" podStartSLOduration=7.6245371760000005 podStartE2EDuration="7.624537176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 21:44:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:44:51.606337058 +0000 UTC m=+7.915566074" watchObservedRunningTime="2026-11-03 21:44:51.624537176 +0000 UTC m=+7.933766190" Nov 03 21:44:51.624804 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:51.624683 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-nz4zw" podStartSLOduration=2.210613228 podStartE2EDuration="7.624676575s" totalImagesPullingTime="5.414063347s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 21:44:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:44:51.624099803 +0000 UTC m=+7.933328829" watchObservedRunningTime="2026-11-03 21:44:51.624676575 +0000 UTC m=+7.933905591" Nov 03 21:44:52.374633 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:52.374594 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrbsd" Nov 03 21:44:52.374811 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:52.374732 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrbsd" podUID="5d7e7316-1bd4-42cf-8519-544c1e2a4526" Nov 03 21:44:52.375197 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:52.375171 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:44:52.375314 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:52.375292 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jq6hj" podUID="94aaa3ac-c7fc-4f87-878e-8ef79127488b" Nov 03 21:44:52.600166 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:52.598832 2578 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Nov 03 21:44:52.600166 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:52.599332 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" event={"ID":"48d8b783-b602-47ef-9b9b-f2416d448908","Type":"ContainerStarted","Data":"61d2bb3a868fbfe3bf0076c61649da5a770503b9cb384027e6e22f244dbcd190"} Nov 03 21:44:52.816366 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:52.816336 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb5f720e-f465-439f-ba32-4f595561f9f6-metrics-certs\") pod \"network-metrics-daemon-knt57\" (UID: \"eb5f720e-f465-439f-ba32-4f595561f9f6\") " pod="openshift-multus/network-metrics-daemon-knt57" Nov 03 21:44:52.816530 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:52.816385 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/94aaa3ac-c7fc-4f87-878e-8ef79127488b-original-pull-secret\") pod \"global-pull-secret-syncer-jq6hj\" (UID: \"94aaa3ac-c7fc-4f87-878e-8ef79127488b\") " pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:44:52.816530 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:52.816478 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 03 21:44:52.816530 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:52.816510 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 03 21:44:52.816530 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:52.816528 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94aaa3ac-c7fc-4f87-878e-8ef79127488b-original-pull-secret podName:94aaa3ac-c7fc-4f87-878e-8ef79127488b nodeName:}" failed. No retries permitted until 2026-11-03 21:45:00.816514281 +0000 UTC m=+17.125743288 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/94aaa3ac-c7fc-4f87-878e-8ef79127488b-original-pull-secret") pod "global-pull-secret-syncer-jq6hj" (UID: "94aaa3ac-c7fc-4f87-878e-8ef79127488b") : object "kube-system"/"original-pull-secret" not registered Nov 03 21:44:52.816766 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:52.816574 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb5f720e-f465-439f-ba32-4f595561f9f6-metrics-certs podName:eb5f720e-f465-439f-ba32-4f595561f9f6 nodeName:}" failed. No retries permitted until 2026-11-03 21:45:00.816557346 +0000 UTC m=+17.125786352 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eb5f720e-f465-439f-ba32-4f595561f9f6-metrics-certs") pod "network-metrics-daemon-knt57" (UID: "eb5f720e-f465-439f-ba32-4f595561f9f6") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 03 21:44:52.917187 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:52.917153 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh4kw\" (UniqueName: \"kubernetes.io/projected/5d7e7316-1bd4-42cf-8519-544c1e2a4526-kube-api-access-zh4kw\") pod \"network-check-target-zrbsd\" (UID: \"5d7e7316-1bd4-42cf-8519-544c1e2a4526\") " pod="openshift-network-diagnostics/network-check-target-zrbsd" Nov 03 21:44:52.917382 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:52.917327 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 03 21:44:52.917382 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:52.917348 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 03 21:44:52.917382 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:52.917358 2578 projected.go:196] Error preparing data for projected volume kube-api-access-zh4kw for pod openshift-network-diagnostics/network-check-target-zrbsd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 03 21:44:52.917557 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:52.917421 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5d7e7316-1bd4-42cf-8519-544c1e2a4526-kube-api-access-zh4kw podName:5d7e7316-1bd4-42cf-8519-544c1e2a4526 nodeName:}" failed. No retries permitted until 2026-11-03 21:45:00.917403582 +0000 UTC m=+17.226632584 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-zh4kw" (UniqueName: "kubernetes.io/projected/5d7e7316-1bd4-42cf-8519-544c1e2a4526-kube-api-access-zh4kw") pod "network-check-target-zrbsd" (UID: "5d7e7316-1bd4-42cf-8519-544c1e2a4526") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 03 21:44:53.372641 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:53.372611 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knt57" Nov 03 21:44:53.372790 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:53.372751 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-knt57" podUID="eb5f720e-f465-439f-ba32-4f595561f9f6" Nov 03 21:44:53.540704 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:53.540673 2578 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Nov 03 21:44:53.607556 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:53.607430 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" event={"ID":"48d8b783-b602-47ef-9b9b-f2416d448908","Type":"ContainerStarted","Data":"a61566bb2117059143219c77a17cf4c052c2f062921b5fd6aa43cf27c0ed61a0"} Nov 03 21:44:54.233529 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:54.232712 2578 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-03T21:44:53.540697159Z","UUID":"9714cc57-a31a-4809-92e3-3e931240b6ee","Handler":null,"Name":"","Endpoint":""} Nov 03 21:44:54.237191 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:54.235086 2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Nov 03 21:44:54.237191 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:54.235117 2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Nov 03 21:44:54.375074 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:54.375040 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:44:54.375232 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:54.375202 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jq6hj" podUID="94aaa3ac-c7fc-4f87-878e-8ef79127488b" Nov 03 21:44:54.375307 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:54.375291 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrbsd" Nov 03 21:44:54.375387 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:54.375371 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrbsd" podUID="5d7e7316-1bd4-42cf-8519-544c1e2a4526" Nov 03 21:44:54.613268 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:54.613181 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mbksx" event={"ID":"219625cd-f1a4-4e51-90ee-58f0fc5939bb","Type":"ContainerStarted","Data":"abad6f50b2d60398e27f7d0fc18d27b6b899f844313f2ec946c6cc57be02afb9"} Nov 03 21:44:54.616534 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:54.616481 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9pctw" event={"ID":"9f5c183e-d58a-4621-940c-04a7ae0bafc9","Type":"ContainerStarted","Data":"b75350cd58117cfee579f0eacb3c50643a1618bca8cfff17c2b7283c0afbe993"} Nov 03 21:44:54.642479 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:54.642422 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-mbksx" podStartSLOduration=2.446074208 podStartE2EDuration="10.642403569s" totalImagesPullingTime="8.196329361s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 21:44:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:44:54.641093597 +0000 UTC m=+10.950322637" watchObservedRunningTime="2026-11-03 21:44:54.642403569 +0000 UTC m=+10.951632582" Nov 03 21:44:55.372721 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:55.372637 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knt57" Nov 03 21:44:55.372861 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:55.372797 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-knt57" podUID="eb5f720e-f465-439f-ba32-4f595561f9f6" Nov 03 21:44:55.622379 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:55.622338 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" event={"ID":"48d8b783-b602-47ef-9b9b-f2416d448908","Type":"ContainerStarted","Data":"be8bcfc08a31f898091365c2d93582a0932ee8bbb6f80f4560e00b7dfdccbb44"} Nov 03 21:44:55.625821 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:55.625220 2578 generic.go:356] "Generic (PLEG): container finished" podID="62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c" containerID="bcfab2b6ff71f13e137f3d54a87305ef838d31642eb49639040928e01b6724c0" exitCode=0 Nov 03 21:44:55.625821 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:55.625286 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4cz4g" event={"ID":"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c","Type":"ContainerDied","Data":"bcfab2b6ff71f13e137f3d54a87305ef838d31642eb49639040928e01b6724c0"} Nov 03 21:44:55.626932 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:55.626899 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nkngm" event={"ID":"0db6d946-a1d8-4459-a883-6f93f7e1182a","Type":"ContainerStarted","Data":"6e6297163745d3b6e76fe9edb6a8bce107c23e33b57b48fab45885eb5904de94"} Nov 03 21:44:55.645110 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:55.645062 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" podStartSLOduration=3.251540624 podStartE2EDuration="11.645048351s" totalImagesPullingTime="8.393507727s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 21:44:44 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:44:55.644619145 +0000 UTC m=+11.953848160" watchObservedRunningTime="2026-11-03 21:44:55.645048351 +0000 UTC m=+11.954277420" Nov 03 21:44:55.645489 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:55.645456 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-9pctw" podStartSLOduration=3.210098866 podStartE2EDuration="11.645444456s" totalImagesPullingTime="8.43534559s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 21:44:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:44:54.664383921 +0000 UTC m=+10.973612937" watchObservedRunningTime="2026-11-03 21:44:55.645444456 +0000 UTC m=+11.954673472" Nov 03 21:44:55.686932 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:55.686882 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-nkngm" podStartSLOduration=3.513536109 podStartE2EDuration="11.686867287s" totalImagesPullingTime="8.173331178s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 21:44:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:44:55.686038849 +0000 UTC m=+11.995267865" watchObservedRunningTime="2026-11-03 21:44:55.686867287 +0000 UTC m=+11.996096306" Nov 03 21:44:56.372467 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:56.372231 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:44:56.372650 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:56.372504 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jq6hj" podUID="94aaa3ac-c7fc-4f87-878e-8ef79127488b" Nov 03 21:44:56.372813 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:56.372231 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrbsd" Nov 03 21:44:56.372922 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:56.372903 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrbsd" podUID="5d7e7316-1bd4-42cf-8519-544c1e2a4526" Nov 03 21:44:56.630049 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:56.629929 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" event={"ID":"9c970a61-3ebf-48c9-a05b-688accd0b18a","Type":"ContainerStarted","Data":"c22019e0f7435534f41e6d66cf102e0b501913ab22091cb29c08ace800478111"} Nov 03 21:44:56.632699 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:56.632671 2578 generic.go:356] "Generic (PLEG): container finished" podID="62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c" containerID="426b25f1f3bca20c5cb8f0743a5754565d6364332133f91c634b44f2a5f08a4c" exitCode=0 Nov 03 21:44:56.632822 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:56.632751 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4cz4g" event={"ID":"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c","Type":"ContainerDied","Data":"426b25f1f3bca20c5cb8f0743a5754565d6364332133f91c634b44f2a5f08a4c"} Nov 03 21:44:56.649955 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:56.649908 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" podStartSLOduration=2.426049073 podStartE2EDuration="12.649895615s" totalImagesPullingTime="10.223846542s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 21:44:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:44:56.649733753 +0000 UTC m=+12.958962780" watchObservedRunningTime="2026-11-03 21:44:56.649895615 +0000 UTC m=+12.959124631" Nov 03 21:44:57.371798 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:57.371768 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knt57" Nov 03 21:44:57.371976 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:57.371903 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-knt57" podUID="eb5f720e-f465-439f-ba32-4f595561f9f6" Nov 03 21:44:58.377665 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:58.377627 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:44:58.378428 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:58.377746 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jq6hj" podUID="94aaa3ac-c7fc-4f87-878e-8ef79127488b" Nov 03 21:44:58.378428 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:58.377802 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrbsd" Nov 03 21:44:58.378428 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:58.377890 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrbsd" podUID="5d7e7316-1bd4-42cf-8519-544c1e2a4526" Nov 03 21:44:58.640796 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:58.640704 2578 generic.go:356] "Generic (PLEG): container finished" podID="62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c" containerID="7d62af7781c487f815f911f6ad53df53abd463176f29148e201de910ee5a05bc" exitCode=0 Nov 03 21:44:58.640796 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:58.640752 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4cz4g" event={"ID":"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c","Type":"ContainerDied","Data":"7d62af7781c487f815f911f6ad53df53abd463176f29148e201de910ee5a05bc"} Nov 03 21:44:59.372241 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:44:59.372206 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knt57" Nov 03 21:44:59.372426 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:44:59.372331 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-knt57" podUID="eb5f720e-f465-439f-ba32-4f595561f9f6" Nov 03 21:45:00.372881 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:00.372683 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrbsd" Nov 03 21:45:00.373285 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:00.372763 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:45:00.373285 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:00.372964 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrbsd" podUID="5d7e7316-1bd4-42cf-8519-544c1e2a4526" Nov 03 21:45:00.373285 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:00.373059 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jq6hj" podUID="94aaa3ac-c7fc-4f87-878e-8ef79127488b" Nov 03 21:45:00.885849 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:00.885824 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb5f720e-f465-439f-ba32-4f595561f9f6-metrics-certs\") pod \"network-metrics-daemon-knt57\" (UID: \"eb5f720e-f465-439f-ba32-4f595561f9f6\") " pod="openshift-multus/network-metrics-daemon-knt57" Nov 03 21:45:00.885954 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:00.885870 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/94aaa3ac-c7fc-4f87-878e-8ef79127488b-original-pull-secret\") pod \"global-pull-secret-syncer-jq6hj\" (UID: \"94aaa3ac-c7fc-4f87-878e-8ef79127488b\") " pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:45:00.886029 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:00.885955 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 03 21:45:00.886029 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:00.886007 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb5f720e-f465-439f-ba32-4f595561f9f6-metrics-certs podName:eb5f720e-f465-439f-ba32-4f595561f9f6 nodeName:}" failed. No retries permitted until 2026-11-03 21:45:16.885992674 +0000 UTC m=+33.195221671 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eb5f720e-f465-439f-ba32-4f595561f9f6-metrics-certs") pod "network-metrics-daemon-knt57" (UID: "eb5f720e-f465-439f-ba32-4f595561f9f6") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 03 21:45:00.886180 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:00.885959 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 03 21:45:00.886180 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:00.886088 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94aaa3ac-c7fc-4f87-878e-8ef79127488b-original-pull-secret podName:94aaa3ac-c7fc-4f87-878e-8ef79127488b nodeName:}" failed. No retries permitted until 2026-11-03 21:45:16.886071428 +0000 UTC m=+33.195300440 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/94aaa3ac-c7fc-4f87-878e-8ef79127488b-original-pull-secret") pod "global-pull-secret-syncer-jq6hj" (UID: "94aaa3ac-c7fc-4f87-878e-8ef79127488b") : object "kube-system"/"original-pull-secret" not registered Nov 03 21:45:00.986869 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:00.986838 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh4kw\" (UniqueName: \"kubernetes.io/projected/5d7e7316-1bd4-42cf-8519-544c1e2a4526-kube-api-access-zh4kw\") pod \"network-check-target-zrbsd\" (UID: \"5d7e7316-1bd4-42cf-8519-544c1e2a4526\") " pod="openshift-network-diagnostics/network-check-target-zrbsd" Nov 03 21:45:00.987052 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:00.987028 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 03 21:45:00.987119 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:00.987063 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 03 21:45:00.987119 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:00.987080 2578 projected.go:196] Error preparing data for projected volume kube-api-access-zh4kw for pod openshift-network-diagnostics/network-check-target-zrbsd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 03 21:45:00.987240 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:00.987166 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5d7e7316-1bd4-42cf-8519-544c1e2a4526-kube-api-access-zh4kw podName:5d7e7316-1bd4-42cf-8519-544c1e2a4526 nodeName:}" failed. No retries permitted until 2026-11-03 21:45:16.987146156 +0000 UTC m=+33.296375161 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-zh4kw" (UniqueName: "kubernetes.io/projected/5d7e7316-1bd4-42cf-8519-544c1e2a4526-kube-api-access-zh4kw") pod "network-check-target-zrbsd" (UID: "5d7e7316-1bd4-42cf-8519-544c1e2a4526") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 03 21:45:01.371884 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:01.371802 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knt57" Nov 03 21:45:01.372026 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:01.371944 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-knt57" podUID="eb5f720e-f465-439f-ba32-4f595561f9f6" Nov 03 21:45:01.650723 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:01.650461 2578 generic.go:356] "Generic (PLEG): container finished" podID="62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c" containerID="da24cd5354d6f06b2fef4b3e4d1d5dffc5044edfc9cd86d18849ca9e9ae7d796" exitCode=0 Nov 03 21:45:01.651221 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:01.650538 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4cz4g" event={"ID":"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c","Type":"ContainerDied","Data":"da24cd5354d6f06b2fef4b3e4d1d5dffc5044edfc9cd86d18849ca9e9ae7d796"} Nov 03 21:45:01.653550 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:01.653529 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x26mj_76122905-15b4-4f4e-97b9-9f73a7675e36/ovn-acl-logging/0.log" Nov 03 21:45:01.653892 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:01.653872 2578 generic.go:356] "Generic (PLEG): container finished" podID="76122905-15b4-4f4e-97b9-9f73a7675e36" containerID="c4b707b05dcedd7bb732861345630a8373dd963050b73d818d0d6e5b79c426e3" exitCode=1 Nov 03 21:45:01.653933 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:01.653912 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" event={"ID":"76122905-15b4-4f4e-97b9-9f73a7675e36","Type":"ContainerStarted","Data":"dd1f50b1dfdbe5866d5f7c51d16d95311caac8bcfb1e323d80304b1c9e334b91"} Nov 03 21:45:01.653987 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:01.653976 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" event={"ID":"76122905-15b4-4f4e-97b9-9f73a7675e36","Type":"ContainerStarted","Data":"b2948e5f49e439fb55271ec96036aedbd5d334a661c8012e1be8537e1353a668"} Nov 03 21:45:01.654018 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:01.654009 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" event={"ID":"76122905-15b4-4f4e-97b9-9f73a7675e36","Type":"ContainerStarted","Data":"f83fbdf9e381384a73791fe9d1abfb59bac74df2dab9878cdc21f6510f1c0c7e"} Nov 03 21:45:01.654051 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:01.654037 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" event={"ID":"76122905-15b4-4f4e-97b9-9f73a7675e36","Type":"ContainerStarted","Data":"18539f8575afa8a277502c35b857e9ee3da8a943d2e20df48a61310d5097306b"} Nov 03 21:45:01.654080 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:01.654066 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" event={"ID":"76122905-15b4-4f4e-97b9-9f73a7675e36","Type":"ContainerDied","Data":"c4b707b05dcedd7bb732861345630a8373dd963050b73d818d0d6e5b79c426e3"} Nov 03 21:45:01.654107 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:01.654094 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" event={"ID":"76122905-15b4-4f4e-97b9-9f73a7675e36","Type":"ContainerStarted","Data":"697e2ef00122620b029e3d3d37268e613dd56d7c111a2b367978522606d5d686"} Nov 03 21:45:02.375517 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:02.375490 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrbsd" Nov 03 21:45:02.375517 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:02.375520 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:45:02.375717 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:02.375594 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrbsd" podUID="5d7e7316-1bd4-42cf-8519-544c1e2a4526" Nov 03 21:45:02.375717 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:02.375642 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jq6hj" podUID="94aaa3ac-c7fc-4f87-878e-8ef79127488b" Nov 03 21:45:02.659072 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:02.658990 2578 generic.go:356] "Generic (PLEG): container finished" podID="62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c" containerID="325011f326827b19ff920f51179dce33712de13818285eb3f64afba0b09d0761" exitCode=0 Nov 03 21:45:02.659072 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:02.659028 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4cz4g" event={"ID":"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c","Type":"ContainerDied","Data":"325011f326827b19ff920f51179dce33712de13818285eb3f64afba0b09d0761"} Nov 03 21:45:03.371810 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:03.371787 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knt57" Nov 03 21:45:03.371941 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:03.371915 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-knt57" podUID="eb5f720e-f465-439f-ba32-4f595561f9f6" Nov 03 21:45:03.664944 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:03.664855 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4cz4g" event={"ID":"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c","Type":"ContainerStarted","Data":"2661256a909887191a1906f5ba14b917c8ec2ba8a0672fc42042e4559e2e461b"} Nov 03 21:45:03.667720 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:03.667700 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x26mj_76122905-15b4-4f4e-97b9-9f73a7675e36/ovn-acl-logging/0.log" Nov 03 21:45:03.668039 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:03.668017 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" event={"ID":"76122905-15b4-4f4e-97b9-9f73a7675e36","Type":"ContainerStarted","Data":"f30e9e4dafc3a74186759ccf1bf559221277aa3990f4fee335bbcd00f75434b3"} Nov 03 21:45:03.700617 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:03.700570 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-4cz4g" podStartSLOduration=8.702387043 podStartE2EDuration="19.700555885s" totalImagesPullingTime="10.998168842s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 21:44:44 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:45:03.700032277 +0000 UTC m=+20.009261291" watchObservedRunningTime="2026-11-03 21:45:03.700555885 +0000 UTC m=+20.009784901" Nov 03 21:45:04.372735 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:04.372702 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:45:04.372904 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:04.372787 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jq6hj" podUID="94aaa3ac-c7fc-4f87-878e-8ef79127488b" Nov 03 21:45:04.372904 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:04.372834 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrbsd" Nov 03 21:45:04.372904 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:04.372879 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrbsd" podUID="5d7e7316-1bd4-42cf-8519-544c1e2a4526" Nov 03 21:45:05.372724 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:05.372693 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knt57" Nov 03 21:45:05.373403 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:05.372817 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-knt57" podUID="eb5f720e-f465-439f-ba32-4f595561f9f6" Nov 03 21:45:05.675010 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:05.674933 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x26mj_76122905-15b4-4f4e-97b9-9f73a7675e36/ovn-acl-logging/0.log" Nov 03 21:45:05.675324 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:05.675302 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" event={"ID":"76122905-15b4-4f4e-97b9-9f73a7675e36","Type":"ContainerStarted","Data":"533c018e7f8091c4d94bb5e891508019611faaf812a485660175fced1e33952e"} Nov 03 21:45:05.675562 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:05.675546 2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:05.675755 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:05.675735 2578 scope.go:122] "RemoveContainer" containerID="c4b707b05dcedd7bb732861345630a8373dd963050b73d818d0d6e5b79c426e3" Nov 03 21:45:05.691660 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:05.691638 2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:06.375175 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:06.375147 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:45:06.375751 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:06.375251 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jq6hj" podUID="94aaa3ac-c7fc-4f87-878e-8ef79127488b" Nov 03 21:45:06.375751 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:06.375299 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrbsd" Nov 03 21:45:06.375751 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:06.375386 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrbsd" podUID="5d7e7316-1bd4-42cf-8519-544c1e2a4526" Nov 03 21:45:06.681035 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:06.680960 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x26mj_76122905-15b4-4f4e-97b9-9f73a7675e36/ovn-acl-logging/0.log" Nov 03 21:45:06.681312 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:06.681291 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" event={"ID":"76122905-15b4-4f4e-97b9-9f73a7675e36","Type":"ContainerStarted","Data":"174aa7930ef67bd1d40770c68d42f54651c0f4a7e3e4759760d4f530ee887a57"} Nov 03 21:45:06.681509 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:06.681491 2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:06.681577 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:06.681557 2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:06.696913 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:06.696888 2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:06.720848 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:06.720805 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" podStartSLOduration=7.730260794 podStartE2EDuration="22.720791478s" totalImagesPullingTime="14.990530684s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 21:44:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:45:06.719611336 +0000 UTC m=+23.028840360" watchObservedRunningTime="2026-11-03 21:45:06.720791478 +0000 UTC m=+23.030020483" Nov 03 21:45:07.372484 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:07.372447 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knt57" Nov 03 21:45:07.372652 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:07.372566 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-knt57" podUID="eb5f720e-f465-439f-ba32-4f595561f9f6" Nov 03 21:45:08.374825 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:08.374792 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrbsd" Nov 03 21:45:08.375390 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:08.374910 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrbsd" podUID="5d7e7316-1bd4-42cf-8519-544c1e2a4526" Nov 03 21:45:08.375390 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:08.374952 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:45:08.375390 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:08.375034 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jq6hj" podUID="94aaa3ac-c7fc-4f87-878e-8ef79127488b" Nov 03 21:45:08.734429 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:08.734361 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9pctw_9f5c183e-d58a-4621-940c-04a7ae0bafc9/dns-node-resolver/0.log" Nov 03 21:45:09.372391 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:09.372358 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knt57" Nov 03 21:45:09.372570 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:09.372480 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-knt57" podUID="eb5f720e-f465-439f-ba32-4f595561f9f6" Nov 03 21:45:09.505783 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:09.505756 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nz4zw_a440580e-b0bd-49e2-a33d-576118e049ae/node-ca/0.log" Nov 03 21:45:10.372162 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:10.372110 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrbsd" Nov 03 21:45:10.372331 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:10.372239 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrbsd" podUID="5d7e7316-1bd4-42cf-8519-544c1e2a4526" Nov 03 21:45:10.372331 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:10.372292 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:45:10.372467 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:10.372373 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jq6hj" podUID="94aaa3ac-c7fc-4f87-878e-8ef79127488b" Nov 03 21:45:11.372268 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:11.372237 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knt57" Nov 03 21:45:11.372723 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:11.372345 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-knt57" podUID="eb5f720e-f465-439f-ba32-4f595561f9f6" Nov 03 21:45:12.372311 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:12.372278 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrbsd" Nov 03 21:45:12.372793 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:12.372399 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrbsd" podUID="5d7e7316-1bd4-42cf-8519-544c1e2a4526" Nov 03 21:45:12.372793 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:12.372444 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:45:12.372793 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:12.372522 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jq6hj" podUID="94aaa3ac-c7fc-4f87-878e-8ef79127488b" Nov 03 21:45:13.372607 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:13.372569 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knt57" Nov 03 21:45:13.373016 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:13.372708 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-knt57" podUID="eb5f720e-f465-439f-ba32-4f595561f9f6" Nov 03 21:45:14.061143 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:14.061100 2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-v98zn" Nov 03 21:45:14.061336 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:14.061233 2578 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Nov 03 21:45:14.061691 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:14.061674 2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-v98zn" Nov 03 21:45:14.373430 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:14.373340 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrbsd" Nov 03 21:45:14.373792 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:14.373437 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrbsd" podUID="5d7e7316-1bd4-42cf-8519-544c1e2a4526" Nov 03 21:45:14.373792 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:14.373513 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:45:14.373792 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:14.373596 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jq6hj" podUID="94aaa3ac-c7fc-4f87-878e-8ef79127488b" Nov 03 21:45:15.372797 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:15.372758 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knt57" Nov 03 21:45:15.373001 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:15.372897 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-knt57" podUID="eb5f720e-f465-439f-ba32-4f595561f9f6" Nov 03 21:45:16.372770 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:16.372733 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrbsd" Nov 03 21:45:16.373288 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:16.372745 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:45:16.373288 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:16.372862 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrbsd" podUID="5d7e7316-1bd4-42cf-8519-544c1e2a4526" Nov 03 21:45:16.373288 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:16.372903 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jq6hj" podUID="94aaa3ac-c7fc-4f87-878e-8ef79127488b" Nov 03 21:45:16.887742 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:16.887717 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb5f720e-f465-439f-ba32-4f595561f9f6-metrics-certs\") pod \"network-metrics-daemon-knt57\" (UID: \"eb5f720e-f465-439f-ba32-4f595561f9f6\") " pod="openshift-multus/network-metrics-daemon-knt57" Nov 03 21:45:16.887936 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:16.887765 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/94aaa3ac-c7fc-4f87-878e-8ef79127488b-original-pull-secret\") pod \"global-pull-secret-syncer-jq6hj\" (UID: \"94aaa3ac-c7fc-4f87-878e-8ef79127488b\") " pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:45:16.887936 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:16.887863 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 03 21:45:16.887936 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:16.887871 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 03 21:45:16.887936 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:16.887913 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94aaa3ac-c7fc-4f87-878e-8ef79127488b-original-pull-secret podName:94aaa3ac-c7fc-4f87-878e-8ef79127488b nodeName:}" failed. No retries permitted until 2026-11-03 21:45:48.887898861 +0000 UTC m=+65.197127858 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/94aaa3ac-c7fc-4f87-878e-8ef79127488b-original-pull-secret") pod "global-pull-secret-syncer-jq6hj" (UID: "94aaa3ac-c7fc-4f87-878e-8ef79127488b") : object "kube-system"/"original-pull-secret" not registered Nov 03 21:45:16.887936 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:16.887926 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb5f720e-f465-439f-ba32-4f595561f9f6-metrics-certs podName:eb5f720e-f465-439f-ba32-4f595561f9f6 nodeName:}" failed. No retries permitted until 2026-11-03 21:45:48.887919949 +0000 UTC m=+65.197148942 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eb5f720e-f465-439f-ba32-4f595561f9f6-metrics-certs") pod "network-metrics-daemon-knt57" (UID: "eb5f720e-f465-439f-ba32-4f595561f9f6") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 03 21:45:16.989028 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:16.988988 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh4kw\" (UniqueName: \"kubernetes.io/projected/5d7e7316-1bd4-42cf-8519-544c1e2a4526-kube-api-access-zh4kw\") pod \"network-check-target-zrbsd\" (UID: \"5d7e7316-1bd4-42cf-8519-544c1e2a4526\") " pod="openshift-network-diagnostics/network-check-target-zrbsd" Nov 03 21:45:16.989245 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:16.989154 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 03 21:45:16.989245 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:16.989176 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 03 21:45:16.989245 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:16.989189 2578 projected.go:196] Error preparing data for projected volume kube-api-access-zh4kw for pod openshift-network-diagnostics/network-check-target-zrbsd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 03 21:45:16.989245 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:16.989234 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5d7e7316-1bd4-42cf-8519-544c1e2a4526-kube-api-access-zh4kw podName:5d7e7316-1bd4-42cf-8519-544c1e2a4526 nodeName:}" failed. No retries permitted until 2026-11-03 21:45:48.989222237 +0000 UTC m=+65.298451235 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-zh4kw" (UniqueName: "kubernetes.io/projected/5d7e7316-1bd4-42cf-8519-544c1e2a4526-kube-api-access-zh4kw") pod "network-check-target-zrbsd" (UID: "5d7e7316-1bd4-42cf-8519-544c1e2a4526") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 03 21:45:17.372548 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:17.372451 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knt57" Nov 03 21:45:17.372702 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:17.372567 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-knt57" podUID="eb5f720e-f465-439f-ba32-4f595561f9f6" Nov 03 21:45:18.372100 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:18.372069 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:45:18.372586 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:18.372206 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jq6hj" podUID="94aaa3ac-c7fc-4f87-878e-8ef79127488b" Nov 03 21:45:18.372586 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:18.372268 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrbsd" Nov 03 21:45:18.372586 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:18.372344 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrbsd" podUID="5d7e7316-1bd4-42cf-8519-544c1e2a4526" Nov 03 21:45:19.372268 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:19.372235 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knt57" Nov 03 21:45:19.372668 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:19.372342 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-knt57" podUID="eb5f720e-f465-439f-ba32-4f595561f9f6" Nov 03 21:45:20.372701 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:20.372660 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:45:20.372701 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:20.372685 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrbsd" Nov 03 21:45:20.373310 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:20.372773 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jq6hj" podUID="94aaa3ac-c7fc-4f87-878e-8ef79127488b" Nov 03 21:45:20.373310 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:20.372885 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrbsd" podUID="5d7e7316-1bd4-42cf-8519-544c1e2a4526" Nov 03 21:45:21.372310 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:21.372274 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knt57" Nov 03 21:45:21.372480 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:21.372387 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-knt57" podUID="eb5f720e-f465-439f-ba32-4f595561f9f6" Nov 03 21:45:22.372497 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:22.372464 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrbsd" Nov 03 21:45:22.372901 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:22.372575 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrbsd" podUID="5d7e7316-1bd4-42cf-8519-544c1e2a4526" Nov 03 21:45:22.372901 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:22.372631 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:45:22.372901 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:22.372745 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jq6hj" podUID="94aaa3ac-c7fc-4f87-878e-8ef79127488b" Nov 03 21:45:23.203863 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:23.203772 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zrbsd"] Nov 03 21:45:23.204019 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:23.203902 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrbsd" Nov 03 21:45:23.204019 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:23.204006 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrbsd" podUID="5d7e7316-1bd4-42cf-8519-544c1e2a4526" Nov 03 21:45:23.210970 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:23.210938 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jq6hj"] Nov 03 21:45:23.211143 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:23.211007 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-knt57"] Nov 03 21:45:23.211143 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:23.211087 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knt57" Nov 03 21:45:23.211261 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:23.211218 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-knt57" podUID="eb5f720e-f465-439f-ba32-4f595561f9f6" Nov 03 21:45:23.211414 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:23.211397 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:45:23.211502 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:23.211486 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jq6hj" podUID="94aaa3ac-c7fc-4f87-878e-8ef79127488b" Nov 03 21:45:25.372199 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:25.372164 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:45:25.372199 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:25.372190 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knt57" Nov 03 21:45:25.372617 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:25.372181 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrbsd" Nov 03 21:45:25.372617 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:25.372286 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jq6hj" podUID="94aaa3ac-c7fc-4f87-878e-8ef79127488b" Nov 03 21:45:25.372617 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:25.372352 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrbsd" podUID="5d7e7316-1bd4-42cf-8519-544c1e2a4526" Nov 03 21:45:25.372617 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:25.372427 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-knt57" podUID="eb5f720e-f465-439f-ba32-4f595561f9f6" Nov 03 21:45:27.372840 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.372603 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knt57" Nov 03 21:45:27.373331 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.372608 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrbsd" Nov 03 21:45:27.373331 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:27.372979 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-knt57" podUID="eb5f720e-f465-439f-ba32-4f595561f9f6" Nov 03 21:45:27.373331 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.372619 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:45:27.373331 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:27.373035 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrbsd" podUID="5d7e7316-1bd4-42cf-8519-544c1e2a4526" Nov 03 21:45:27.373331 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:27.373074 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jq6hj" podUID="94aaa3ac-c7fc-4f87-878e-8ef79127488b" Nov 03 21:45:27.508275 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.508246 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-7.us-east-2.compute.internal" event="NodeReady" Nov 03 21:45:27.508431 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.508380 2578 kubelet_node_status.go:477] "Fast updating node status as it just became ready" Nov 03 21:45:27.557271 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.557196 2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-x2r5s"] Nov 03 21:45:27.559045 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.559026 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-x2r5s" Nov 03 21:45:27.560264 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.560238 2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-285hg"] Nov 03 21:45:27.561866 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.561849 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-285hg" Nov 03 21:45:27.564539 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.564515 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Nov 03 21:45:27.564634 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.564526 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Nov 03 21:45:27.564689 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.564656 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-92tpk\"" Nov 03 21:45:27.571150 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.571113 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-x2r5s"] Nov 03 21:45:27.574440 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.574020 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-285hg"] Nov 03 21:45:27.659264 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.659234 2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-77wdg"] Nov 03 21:45:27.660939 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.660914 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-77wdg" Nov 03 21:45:27.664757 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.664660 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Nov 03 21:45:27.664917 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.664720 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Nov 03 21:45:27.664917 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.664724 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Nov 03 21:45:27.664917 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.664727 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Nov 03 21:45:27.664917 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.664759 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5921d9cf-54a7-43bd-b697-c5ccf9f39135-config-volume\") pod \"dns-default-x2r5s\" (UID: \"5921d9cf-54a7-43bd-b697-c5ccf9f39135\") " pod="openshift-dns/dns-default-x2r5s" Nov 03 21:45:27.664917 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.664915 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Nov 03 21:45:27.665185 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.664914 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e652c3ec-d5ac-43f9-ade5-d3da32041cee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-285hg\" (UID: \"e652c3ec-d5ac-43f9-ade5-d3da32041cee\") " pod="openshift-insights/insights-runtime-extractor-285hg" Nov 03 21:45:27.665185 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.664929 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Nov 03 21:45:27.665185 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.664948 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-7bvj7\"" Nov 03 21:45:27.665185 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.664987 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Nov 03 21:45:27.665185 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.664987 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7m8x\" (UniqueName: \"kubernetes.io/projected/5921d9cf-54a7-43bd-b697-c5ccf9f39135-kube-api-access-t7m8x\") pod \"dns-default-x2r5s\" (UID: \"5921d9cf-54a7-43bd-b697-c5ccf9f39135\") " pod="openshift-dns/dns-default-x2r5s" Nov 03 21:45:27.665185 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.665003 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-wc5kn\"" Nov 03 21:45:27.665185 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.665032 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e652c3ec-d5ac-43f9-ade5-d3da32041cee-data-volume\") pod \"insights-runtime-extractor-285hg\" (UID: \"e652c3ec-d5ac-43f9-ade5-d3da32041cee\") " pod="openshift-insights/insights-runtime-extractor-285hg" Nov 03 21:45:27.665185 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.665087 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7xhd\" (UniqueName: \"kubernetes.io/projected/e652c3ec-d5ac-43f9-ade5-d3da32041cee-kube-api-access-p7xhd\") pod \"insights-runtime-extractor-285hg\" (UID: \"e652c3ec-d5ac-43f9-ade5-d3da32041cee\") " pod="openshift-insights/insights-runtime-extractor-285hg" Nov 03 21:45:27.665185 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.665159 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e652c3ec-d5ac-43f9-ade5-d3da32041cee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-285hg\" (UID: \"e652c3ec-d5ac-43f9-ade5-d3da32041cee\") " pod="openshift-insights/insights-runtime-extractor-285hg" Nov 03 21:45:27.665185 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.665189 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5921d9cf-54a7-43bd-b697-c5ccf9f39135-metrics-tls\") pod \"dns-default-x2r5s\" (UID: \"5921d9cf-54a7-43bd-b697-c5ccf9f39135\") " pod="openshift-dns/dns-default-x2r5s" Nov 03 21:45:27.665554 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.665233 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5921d9cf-54a7-43bd-b697-c5ccf9f39135-tmp-dir\") pod \"dns-default-x2r5s\" (UID: \"5921d9cf-54a7-43bd-b697-c5ccf9f39135\") " pod="openshift-dns/dns-default-x2r5s" Nov 03 21:45:27.665554 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.665299 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e652c3ec-d5ac-43f9-ade5-d3da32041cee-crio-socket\") pod \"insights-runtime-extractor-285hg\" (UID: \"e652c3ec-d5ac-43f9-ade5-d3da32041cee\") " pod="openshift-insights/insights-runtime-extractor-285hg" Nov 03 21:45:27.671494 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.671472 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-77wdg"] Nov 03 21:45:27.766077 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.766034 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7xhd\" (UniqueName: \"kubernetes.io/projected/e652c3ec-d5ac-43f9-ade5-d3da32041cee-kube-api-access-p7xhd\") pod \"insights-runtime-extractor-285hg\" (UID: \"e652c3ec-d5ac-43f9-ade5-d3da32041cee\") " pod="openshift-insights/insights-runtime-extractor-285hg" Nov 03 21:45:27.766300 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.766084 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e652c3ec-d5ac-43f9-ade5-d3da32041cee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-285hg\" (UID: \"e652c3ec-d5ac-43f9-ade5-d3da32041cee\") " pod="openshift-insights/insights-runtime-extractor-285hg" Nov 03 21:45:27.766300 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.766193 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5921d9cf-54a7-43bd-b697-c5ccf9f39135-metrics-tls\") pod \"dns-default-x2r5s\" (UID: \"5921d9cf-54a7-43bd-b697-c5ccf9f39135\") " pod="openshift-dns/dns-default-x2r5s" Nov 03 21:45:27.766300 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.766268 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/32fd1d86-45ed-4a7d-894e-3990980bc237-cert\") pod \"ingress-canary-77wdg\" (UID: \"32fd1d86-45ed-4a7d-894e-3990980bc237\") " pod="openshift-ingress-canary/ingress-canary-77wdg" Nov 03 21:45:27.766458 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.766316 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5921d9cf-54a7-43bd-b697-c5ccf9f39135-tmp-dir\") pod \"dns-default-x2r5s\" (UID: \"5921d9cf-54a7-43bd-b697-c5ccf9f39135\") " pod="openshift-dns/dns-default-x2r5s" Nov 03 21:45:27.766458 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.766340 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e652c3ec-d5ac-43f9-ade5-d3da32041cee-crio-socket\") pod \"insights-runtime-extractor-285hg\" (UID: \"e652c3ec-d5ac-43f9-ade5-d3da32041cee\") " pod="openshift-insights/insights-runtime-extractor-285hg" Nov 03 21:45:27.766538 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.766487 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5921d9cf-54a7-43bd-b697-c5ccf9f39135-config-volume\") pod \"dns-default-x2r5s\" (UID: \"5921d9cf-54a7-43bd-b697-c5ccf9f39135\") " pod="openshift-dns/dns-default-x2r5s" Nov 03 21:45:27.766538 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.766530 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8h9g\" (UniqueName: \"kubernetes.io/projected/32fd1d86-45ed-4a7d-894e-3990980bc237-kube-api-access-m8h9g\") pod \"ingress-canary-77wdg\" (UID: \"32fd1d86-45ed-4a7d-894e-3990980bc237\") " pod="openshift-ingress-canary/ingress-canary-77wdg" Nov 03 21:45:27.766635 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.766544 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e652c3ec-d5ac-43f9-ade5-d3da32041cee-crio-socket\") pod \"insights-runtime-extractor-285hg\" (UID: \"e652c3ec-d5ac-43f9-ade5-d3da32041cee\") " pod="openshift-insights/insights-runtime-extractor-285hg" Nov 03 21:45:27.766696 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.766672 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e652c3ec-d5ac-43f9-ade5-d3da32041cee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-285hg\" (UID: \"e652c3ec-d5ac-43f9-ade5-d3da32041cee\") " pod="openshift-insights/insights-runtime-extractor-285hg" Nov 03 21:45:27.766739 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.766728 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5921d9cf-54a7-43bd-b697-c5ccf9f39135-tmp-dir\") pod \"dns-default-x2r5s\" (UID: \"5921d9cf-54a7-43bd-b697-c5ccf9f39135\") " pod="openshift-dns/dns-default-x2r5s" Nov 03 21:45:27.766782 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.766740 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7m8x\" (UniqueName: \"kubernetes.io/projected/5921d9cf-54a7-43bd-b697-c5ccf9f39135-kube-api-access-t7m8x\") pod \"dns-default-x2r5s\" (UID: \"5921d9cf-54a7-43bd-b697-c5ccf9f39135\") " pod="openshift-dns/dns-default-x2r5s" Nov 03 21:45:27.766782 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.766772 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e652c3ec-d5ac-43f9-ade5-d3da32041cee-data-volume\") pod \"insights-runtime-extractor-285hg\" (UID: \"e652c3ec-d5ac-43f9-ade5-d3da32041cee\") " pod="openshift-insights/insights-runtime-extractor-285hg" Nov 03 21:45:27.767086 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.767063 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e652c3ec-d5ac-43f9-ade5-d3da32041cee-data-volume\") pod \"insights-runtime-extractor-285hg\" (UID: \"e652c3ec-d5ac-43f9-ade5-d3da32041cee\") " pod="openshift-insights/insights-runtime-extractor-285hg" Nov 03 21:45:27.767086 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.767079 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5921d9cf-54a7-43bd-b697-c5ccf9f39135-config-volume\") pod \"dns-default-x2r5s\" (UID: \"5921d9cf-54a7-43bd-b697-c5ccf9f39135\") " pod="openshift-dns/dns-default-x2r5s" Nov 03 21:45:27.767277 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.767063 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e652c3ec-d5ac-43f9-ade5-d3da32041cee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-285hg\" (UID: \"e652c3ec-d5ac-43f9-ade5-d3da32041cee\") " pod="openshift-insights/insights-runtime-extractor-285hg" Nov 03 21:45:27.770191 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.770164 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5921d9cf-54a7-43bd-b697-c5ccf9f39135-metrics-tls\") pod \"dns-default-x2r5s\" (UID: \"5921d9cf-54a7-43bd-b697-c5ccf9f39135\") " pod="openshift-dns/dns-default-x2r5s" Nov 03 21:45:27.770191 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.770169 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e652c3ec-d5ac-43f9-ade5-d3da32041cee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-285hg\" (UID: \"e652c3ec-d5ac-43f9-ade5-d3da32041cee\") " pod="openshift-insights/insights-runtime-extractor-285hg" Nov 03 21:45:27.778260 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.778232 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7xhd\" (UniqueName: \"kubernetes.io/projected/e652c3ec-d5ac-43f9-ade5-d3da32041cee-kube-api-access-p7xhd\") pod \"insights-runtime-extractor-285hg\" (UID: \"e652c3ec-d5ac-43f9-ade5-d3da32041cee\") " pod="openshift-insights/insights-runtime-extractor-285hg" Nov 03 21:45:27.778342 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.778272 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7m8x\" (UniqueName: \"kubernetes.io/projected/5921d9cf-54a7-43bd-b697-c5ccf9f39135-kube-api-access-t7m8x\") pod \"dns-default-x2r5s\" (UID: \"5921d9cf-54a7-43bd-b697-c5ccf9f39135\") " pod="openshift-dns/dns-default-x2r5s" Nov 03 21:45:27.867871 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.867835 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8h9g\" (UniqueName: \"kubernetes.io/projected/32fd1d86-45ed-4a7d-894e-3990980bc237-kube-api-access-m8h9g\") pod \"ingress-canary-77wdg\" (UID: \"32fd1d86-45ed-4a7d-894e-3990980bc237\") " pod="openshift-ingress-canary/ingress-canary-77wdg" Nov 03 21:45:27.868042 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.867898 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/32fd1d86-45ed-4a7d-894e-3990980bc237-cert\") pod \"ingress-canary-77wdg\" (UID: \"32fd1d86-45ed-4a7d-894e-3990980bc237\") " pod="openshift-ingress-canary/ingress-canary-77wdg" Nov 03 21:45:27.870228 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.870207 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/32fd1d86-45ed-4a7d-894e-3990980bc237-cert\") pod \"ingress-canary-77wdg\" (UID: \"32fd1d86-45ed-4a7d-894e-3990980bc237\") " pod="openshift-ingress-canary/ingress-canary-77wdg" Nov 03 21:45:27.872066 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.872045 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-x2r5s" Nov 03 21:45:27.880591 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.880575 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8h9g\" (UniqueName: \"kubernetes.io/projected/32fd1d86-45ed-4a7d-894e-3990980bc237-kube-api-access-m8h9g\") pod \"ingress-canary-77wdg\" (UID: \"32fd1d86-45ed-4a7d-894e-3990980bc237\") " pod="openshift-ingress-canary/ingress-canary-77wdg" Nov 03 21:45:27.882377 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.882362 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-285hg" Nov 03 21:45:27.971535 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:27.971506 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-77wdg" Nov 03 21:45:28.167048 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:28.166973 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-x2r5s"] Nov 03 21:45:28.169849 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:28.169815 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-285hg"] Nov 03 21:45:28.172635 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:45:28.172610 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode652c3ec_d5ac_43f9_ade5_d3da32041cee.slice/crio-06948bda24b584813e431ecca9ab2c381f1466d883d2c21d599a46a87854cb4d WatchSource:0}: Error finding container 06948bda24b584813e431ecca9ab2c381f1466d883d2c21d599a46a87854cb4d: Status 404 returned error can't find the container with id 06948bda24b584813e431ecca9ab2c381f1466d883d2c21d599a46a87854cb4d Nov 03 21:45:28.242491 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:28.242469 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-77wdg"] Nov 03 21:45:28.245485 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:45:28.245462 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32fd1d86_45ed_4a7d_894e_3990980bc237.slice/crio-5909793b2fc1d44e499b7ee12a8b1b1ff8d06a17a4790707c3eb3ab97c2259f0 WatchSource:0}: Error finding container 5909793b2fc1d44e499b7ee12a8b1b1ff8d06a17a4790707c3eb3ab97c2259f0: Status 404 returned error can't find the container with id 5909793b2fc1d44e499b7ee12a8b1b1ff8d06a17a4790707c3eb3ab97c2259f0 Nov 03 21:45:28.521932 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:28.521808 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9pctw_9f5c183e-d58a-4621-940c-04a7ae0bafc9/dns-node-resolver/0.log" Nov 03 21:45:28.730978 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:28.730940 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-285hg" event={"ID":"e652c3ec-d5ac-43f9-ade5-d3da32041cee","Type":"ContainerStarted","Data":"05d6b1a346b027ce45af4ca5247fbc3469e58f6ed7361173e8cdab77cdc80014"} Nov 03 21:45:28.731161 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:28.731004 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-285hg" event={"ID":"e652c3ec-d5ac-43f9-ade5-d3da32041cee","Type":"ContainerStarted","Data":"06948bda24b584813e431ecca9ab2c381f1466d883d2c21d599a46a87854cb4d"} Nov 03 21:45:28.731944 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:28.731924 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-77wdg" event={"ID":"32fd1d86-45ed-4a7d-894e-3990980bc237","Type":"ContainerStarted","Data":"5909793b2fc1d44e499b7ee12a8b1b1ff8d06a17a4790707c3eb3ab97c2259f0"} Nov 03 21:45:28.733132 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:28.733099 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x2r5s" event={"ID":"5921d9cf-54a7-43bd-b697-c5ccf9f39135","Type":"ContainerStarted","Data":"9d446131f627080dcfe0c7098bf0cc9b0e356964bd3cca77e96faf8eba808bb3"} Nov 03 21:45:29.372188 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:29.372156 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knt57" Nov 03 21:45:29.372370 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:29.372163 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrbsd" Nov 03 21:45:29.372638 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:29.372174 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:45:29.376429 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:29.376368 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Nov 03 21:45:29.377853 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:29.377831 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-p72km\"" Nov 03 21:45:29.377853 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:29.377847 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Nov 03 21:45:29.378003 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:29.377834 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Nov 03 21:45:29.378157 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:29.378096 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-t98zr\"" Nov 03 21:45:29.378157 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:29.378105 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-5xcnq\"" Nov 03 21:45:29.378157 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:29.378097 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Nov 03 21:45:29.523758 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:29.523725 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nz4zw_a440580e-b0bd-49e2-a33d-576118e049ae/node-ca/0.log" Nov 03 21:45:29.740894 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:29.740812 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-285hg" event={"ID":"e652c3ec-d5ac-43f9-ade5-d3da32041cee","Type":"ContainerStarted","Data":"51c525e74c1705e7c278ee24650d37c67c39df9fbfd396b8f6bd3bbd3167f0c5"} Nov 03 21:45:30.371687 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:30.371657 2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-jdbs4"] Nov 03 21:45:30.374819 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:30.374793 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:30.438943 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:30.438916 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Nov 03 21:45:30.439072 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:30.438991 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Nov 03 21:45:30.439218 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:30.439200 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-s5f4x\"" Nov 03 21:45:30.457745 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:30.457719 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Nov 03 21:45:30.458189 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:30.458045 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Nov 03 21:45:30.458492 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:30.458135 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Nov 03 21:45:30.458768 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:30.458229 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Nov 03 21:45:30.489249 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:30.488971 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-sys\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:30.489249 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:30.489018 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-root\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:30.489249 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:30.489058 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-node-exporter-wtmp\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:30.489249 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:30.489104 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:30.490293 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:30.489951 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jl4cx\" (UniqueName: \"kubernetes.io/projected/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-kube-api-access-jl4cx\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:30.490293 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:30.490001 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-node-exporter-textfile\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:30.490293 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:30.490028 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-node-exporter-tls\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:30.490293 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:30.490052 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-metrics-client-ca\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:30.490293 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:30.490089 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-node-exporter-accelerators-collector-config\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:30.590501 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:30.590466 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jl4cx\" (UniqueName: \"kubernetes.io/projected/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-kube-api-access-jl4cx\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:30.590969 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:30.590512 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-node-exporter-textfile\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:30.590969 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:30.590643 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-node-exporter-tls\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:30.590969 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:30.590685 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-metrics-client-ca\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:30.590969 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:30.590726 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-node-exporter-accelerators-collector-config\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:30.590969 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:30.590790 2578 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Nov 03 21:45:30.590969 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:30.590802 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-sys\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:30.590969 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:30.590826 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-root\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:30.590969 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:30.590846 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-node-exporter-tls podName:bfd8df86-d3d0-4576-afd0-fa742a6a3aad nodeName:}" failed. No retries permitted until 2026-11-03 21:45:31.090827088 +0000 UTC m=+47.400056095 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-node-exporter-tls") pod "node-exporter-jdbs4" (UID: "bfd8df86-d3d0-4576-afd0-fa742a6a3aad") : secret "node-exporter-tls" not found Nov 03 21:45:30.590969 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:30.590881 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-root\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:30.590969 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:30.590895 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-node-exporter-wtmp\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:30.590969 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:30.590933 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:30.591436 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:30.591029 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-node-exporter-textfile\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:30.591436 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:30.591181 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-node-exporter-wtmp\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:30.591436 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:30.591238 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-sys\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:30.591534 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:30.591489 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-metrics-client-ca\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:30.591534 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:30.591512 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-node-exporter-accelerators-collector-config\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:30.594382 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:30.594357 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:30.604446 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:30.604393 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jl4cx\" (UniqueName: \"kubernetes.io/projected/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-kube-api-access-jl4cx\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:30.746292 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:30.746254 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x2r5s" event={"ID":"5921d9cf-54a7-43bd-b697-c5ccf9f39135","Type":"ContainerStarted","Data":"095ad7bd0495d7193bf37a7ea21bf3bcef403a60b427a464996f9e70944247d0"} Nov 03 21:45:30.746469 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:30.746354 2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-x2r5s" Nov 03 21:45:30.746469 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:30.746379 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x2r5s" event={"ID":"5921d9cf-54a7-43bd-b697-c5ccf9f39135","Type":"ContainerStarted","Data":"4bf75c4d3bde2ab6e630052bcacecc30c63366db33971734747403fec0bfd28a"} Nov 03 21:45:30.749110 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:30.749082 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-285hg" event={"ID":"e652c3ec-d5ac-43f9-ade5-d3da32041cee","Type":"ContainerStarted","Data":"a859c89255a7cf266e8724d544539091306e243a7da35e790e28aeda5c47b3d5"} Nov 03 21:45:30.802645 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:30.802153 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-x2r5s" podStartSLOduration=1.628028301 podStartE2EDuration="3.802112851s" totalImagesPullingTime="2.17408455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:45:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:45:30.800943528 +0000 UTC m=+47.110172578" watchObservedRunningTime="2026-11-03 21:45:30.802112851 +0000 UTC m=+47.111341867" Nov 03 21:45:30.824692 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:30.824626 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-285hg" podStartSLOduration=1.6278872130000002 podStartE2EDuration="3.824605981s" totalImagesPullingTime="2.196718768s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 21:45:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:45:30.823606724 +0000 UTC m=+47.132835741" watchObservedRunningTime="2026-11-03 21:45:30.824605981 +0000 UTC m=+47.133835006" Nov 03 21:45:31.095720 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.095692 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-node-exporter-tls\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:31.097874 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.097846 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-node-exporter-tls\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:31.223240 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.223208 2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-86cf959768-tf4fd"] Nov 03 21:45:31.225051 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.225032 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:31.229842 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.229812 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Nov 03 21:45:31.229842 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.229838 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Nov 03 21:45:31.230039 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.229887 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Nov 03 21:45:31.230039 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.229968 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Nov 03 21:45:31.230039 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.230012 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Nov 03 21:45:31.230721 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.230700 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Nov 03 21:45:31.230840 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.230725 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Nov 03 21:45:31.230840 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.230818 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-nmvdk\"" Nov 03 21:45:31.233917 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.233900 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Nov 03 21:45:31.237656 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.237629 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86cf959768-tf4fd"] Nov 03 21:45:31.290768 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.290732 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:31.300806 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:45:31.300782 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfd8df86_d3d0_4576_afd0_fa742a6a3aad.slice/crio-0b349470d1bf258e06b2810cc311aff71f9653d22e5f6598ec3f038f6646a0bb WatchSource:0}: Error finding container 0b349470d1bf258e06b2810cc311aff71f9653d22e5f6598ec3f038f6646a0bb: Status 404 returned error can't find the container with id 0b349470d1bf258e06b2810cc311aff71f9653d22e5f6598ec3f038f6646a0bb Nov 03 21:45:31.397991 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.397907 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-oauth-serving-cert\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:31.397991 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.397963 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5kf4\" (UniqueName: \"kubernetes.io/projected/866ddb5a-c929-40c1-911d-ca40c5405723-kube-api-access-n5kf4\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:31.398196 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.398056 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-console-config\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:31.398196 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.398087 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-service-ca\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:31.398196 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.398108 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-oauth-config\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:31.398196 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.398156 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/866ddb5a-c929-40c1-911d-ca40c5405723-tmp\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:31.398322 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.398281 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-serving-cert\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:31.398322 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.398306 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-trusted-ca-bundle\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:31.455708 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.455672 2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 03 21:45:31.458571 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.458554 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:31.463112 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.462971 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Nov 03 21:45:31.463112 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.462979 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Nov 03 21:45:31.463112 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.463010 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Nov 03 21:45:31.463112 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.463010 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Nov 03 21:45:31.463112 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.463037 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Nov 03 21:45:31.463112 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.463071 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Nov 03 21:45:31.463112 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.463074 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Nov 03 21:45:31.463112 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.463075 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Nov 03 21:45:31.463112 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.463090 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Nov 03 21:45:31.463624 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.463168 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-8vqcv\"" Nov 03 21:45:31.479050 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.479023 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 03 21:45:31.498643 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.498610 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-oauth-serving-cert\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:31.498800 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.498654 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5kf4\" (UniqueName: \"kubernetes.io/projected/866ddb5a-c929-40c1-911d-ca40c5405723-kube-api-access-n5kf4\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:31.498800 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.498684 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-console-config\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:31.498800 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.498706 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-service-ca\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:31.498800 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.498736 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-oauth-config\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:31.498800 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.498761 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/866ddb5a-c929-40c1-911d-ca40c5405723-tmp\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:31.499047 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.498833 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-serving-cert\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:31.499047 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.498854 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-trusted-ca-bundle\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:31.499346 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.499307 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/866ddb5a-c929-40c1-911d-ca40c5405723-tmp\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:31.499535 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.499480 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-oauth-serving-cert\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:31.499535 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.499504 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-console-config\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:31.499695 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.499614 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-service-ca\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:31.499695 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.499636 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-trusted-ca-bundle\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:31.501262 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.501239 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-oauth-config\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:31.501326 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.501315 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-serving-cert\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:31.514020 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.513996 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5kf4\" (UniqueName: \"kubernetes.io/projected/866ddb5a-c929-40c1-911d-ca40c5405723-kube-api-access-n5kf4\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:31.535674 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.535639 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:31.599714 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.599683 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7g58\" (UniqueName: \"kubernetes.io/projected/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-kube-api-access-s7g58\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:31.600154 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.599727 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:31.600154 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.599767 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:31.600154 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.599848 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-config-out\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:31.600154 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.599877 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:31.600154 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.599924 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-config-volume\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:31.600154 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.599949 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:31.600154 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.600018 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-web-config\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:31.600154 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.600059 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:31.600154 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.600091 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:31.600154 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.600146 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:31.600607 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.600168 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:31.600607 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.600199 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:31.700678 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.700581 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-config-volume\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:31.700678 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.700626 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:31.700907 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.700783 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-web-config\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:31.700907 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.700825 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:31.700907 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.700860 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:31.700907 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.700899 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:31.701097 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.700924 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:31.701097 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.700957 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:31.701097 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.701008 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7g58\" (UniqueName: \"kubernetes.io/projected/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-kube-api-access-s7g58\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:31.701097 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.701036 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:31.701097 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.701060 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:31.701350 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.701115 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-config-out\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:31.701350 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.701168 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:31.701350 ip-10-0-137-7 kubenswrapper[2578]: E1103 21:45:31.701316 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-alertmanager-trusted-ca-bundle podName:a2ccde5e-94a2-47ac-aff9-65248e4a6c92 nodeName:}" failed. No retries permitted until 2026-11-03 21:45:32.201298289 +0000 UTC m=+48.510527302 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "a2ccde5e-94a2-47ac-aff9-65248e4a6c92") : configmap references non-existent config key: ca-bundle.crt Nov 03 21:45:31.701495 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.701446 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:31.701917 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.701876 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:31.704024 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.703976 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-config-volume\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:31.704024 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.703974 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-config-out\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:31.704279 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.704257 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:31.704338 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.704327 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:31.704429 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.704404 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:31.704512 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.704491 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:31.704548 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.704502 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:31.704799 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.704774 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-web-config\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:31.705598 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.705580 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:31.713784 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.713756 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7g58\" (UniqueName: \"kubernetes.io/projected/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-kube-api-access-s7g58\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:31.753509 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.753465 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-77wdg" event={"ID":"32fd1d86-45ed-4a7d-894e-3990980bc237","Type":"ContainerStarted","Data":"d8ca17dba0f47f4b788b2d385ff52738e48f1e21a7aa6b2eae03b7ace8514b83"} Nov 03 21:45:31.754828 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.754790 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jdbs4" event={"ID":"bfd8df86-d3d0-4576-afd0-fa742a6a3aad","Type":"ContainerStarted","Data":"0b349470d1bf258e06b2810cc311aff71f9653d22e5f6598ec3f038f6646a0bb"} Nov 03 21:45:31.772090 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.772029 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-77wdg" podStartSLOduration=1.9071859880000002 podStartE2EDuration="4.772016148s" totalImagesPullingTime="2.86483016s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:45:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:45:31.771796397 +0000 UTC m=+48.081025414" watchObservedRunningTime="2026-11-03 21:45:31.772016148 +0000 UTC m=+48.081245163" Nov 03 21:45:31.817595 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:31.817559 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86cf959768-tf4fd"] Nov 03 21:45:31.820854 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:45:31.820773 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod866ddb5a_c929_40c1_911d_ca40c5405723.slice/crio-ecfdcbf875981eb0a48d60cbc1003e95e865da3fd3fd6dec53c35397b6308e43 WatchSource:0}: Error finding container ecfdcbf875981eb0a48d60cbc1003e95e865da3fd3fd6dec53c35397b6308e43: Status 404 returned error can't find the container with id ecfdcbf875981eb0a48d60cbc1003e95e865da3fd3fd6dec53c35397b6308e43 Nov 03 21:45:32.205049 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.205011 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:32.205677 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.205657 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:32.374829 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.374802 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:32.443154 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.431854 2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-54674f497-2g5ct"] Nov 03 21:45:32.443154 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.435750 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:32.451768 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.451729 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-54674f497-2g5ct"] Nov 03 21:45:32.461630 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.461513 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Nov 03 21:45:32.461630 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.461529 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Nov 03 21:45:32.461630 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.461548 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Nov 03 21:45:32.461630 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.461536 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-c54pc\"" Nov 03 21:45:32.461918 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.461672 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Nov 03 21:45:32.461918 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.461697 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-5s11ulothan92\"" Nov 03 21:45:32.461918 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.461824 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Nov 03 21:45:32.608652 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.608612 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/00c924c1-ac0d-49c8-8610-3b837fbb8277-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:32.608652 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.608657 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00c924c1-ac0d-49c8-8610-3b837fbb8277-tmp\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:32.609193 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.608700 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phmrg\" (UniqueName: \"kubernetes.io/projected/00c924c1-ac0d-49c8-8610-3b837fbb8277-kube-api-access-phmrg\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:32.609193 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.608811 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/00c924c1-ac0d-49c8-8610-3b837fbb8277-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:32.609193 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.608832 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00c924c1-ac0d-49c8-8610-3b837fbb8277-metrics-client-ca\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:32.609193 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.608860 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/00c924c1-ac0d-49c8-8610-3b837fbb8277-secret-thanos-querier-tls\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:32.609193 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.608886 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/00c924c1-ac0d-49c8-8610-3b837fbb8277-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:32.609193 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.608906 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/00c924c1-ac0d-49c8-8610-3b837fbb8277-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:32.609193 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.608923 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/00c924c1-ac0d-49c8-8610-3b837fbb8277-secret-grpc-tls\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:32.664880 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.664849 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 03 21:45:32.668661 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:45:32.668634 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2ccde5e_94a2_47ac_aff9_65248e4a6c92.slice/crio-106c43f9fcf1a1c3acc5d75e8010b45678a9a16af360e8ab411f4c897fd8dbc2 WatchSource:0}: Error finding container 106c43f9fcf1a1c3acc5d75e8010b45678a9a16af360e8ab411f4c897fd8dbc2: Status 404 returned error can't find the container with id 106c43f9fcf1a1c3acc5d75e8010b45678a9a16af360e8ab411f4c897fd8dbc2 Nov 03 21:45:32.709952 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.709913 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/00c924c1-ac0d-49c8-8610-3b837fbb8277-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:32.709952 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.709944 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/00c924c1-ac0d-49c8-8610-3b837fbb8277-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:32.710197 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.709962 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/00c924c1-ac0d-49c8-8610-3b837fbb8277-secret-grpc-tls\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:32.710197 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.709982 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/00c924c1-ac0d-49c8-8610-3b837fbb8277-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:32.710197 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.710002 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00c924c1-ac0d-49c8-8610-3b837fbb8277-tmp\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:32.710197 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.710040 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phmrg\" (UniqueName: \"kubernetes.io/projected/00c924c1-ac0d-49c8-8610-3b837fbb8277-kube-api-access-phmrg\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:32.710197 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.710086 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/00c924c1-ac0d-49c8-8610-3b837fbb8277-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:32.710197 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.710157 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00c924c1-ac0d-49c8-8610-3b837fbb8277-metrics-client-ca\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:32.710470 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.710221 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/00c924c1-ac0d-49c8-8610-3b837fbb8277-secret-thanos-querier-tls\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:32.711054 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.710898 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00c924c1-ac0d-49c8-8610-3b837fbb8277-tmp\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:32.711054 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.710974 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00c924c1-ac0d-49c8-8610-3b837fbb8277-metrics-client-ca\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:32.712853 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.712797 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/00c924c1-ac0d-49c8-8610-3b837fbb8277-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:32.713035 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.713012 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/00c924c1-ac0d-49c8-8610-3b837fbb8277-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:32.713525 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.713503 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/00c924c1-ac0d-49c8-8610-3b837fbb8277-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:32.713972 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.713950 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/00c924c1-ac0d-49c8-8610-3b837fbb8277-secret-thanos-querier-tls\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:32.714163 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.714143 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/00c924c1-ac0d-49c8-8610-3b837fbb8277-secret-grpc-tls\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:32.714406 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.714383 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/00c924c1-ac0d-49c8-8610-3b837fbb8277-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:32.723441 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.723417 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-phmrg\" (UniqueName: \"kubernetes.io/projected/00c924c1-ac0d-49c8-8610-3b837fbb8277-kube-api-access-phmrg\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:32.757450 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.757419 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:32.760211 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.760189 2578 generic.go:356] "Generic (PLEG): container finished" podID="bfd8df86-d3d0-4576-afd0-fa742a6a3aad" containerID="4d61365efa6dfaf176e3e5455b1ab86a931077e22613ce4fc29799f8e459d000" exitCode=0 Nov 03 21:45:32.760297 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.760273 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jdbs4" event={"ID":"bfd8df86-d3d0-4576-afd0-fa742a6a3aad","Type":"ContainerDied","Data":"4d61365efa6dfaf176e3e5455b1ab86a931077e22613ce4fc29799f8e459d000"} Nov 03 21:45:32.761768 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.761741 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a2ccde5e-94a2-47ac-aff9-65248e4a6c92","Type":"ContainerStarted","Data":"106c43f9fcf1a1c3acc5d75e8010b45678a9a16af360e8ab411f4c897fd8dbc2"} Nov 03 21:45:32.763268 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:32.763247 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86cf959768-tf4fd" event={"ID":"866ddb5a-c929-40c1-911d-ca40c5405723","Type":"ContainerStarted","Data":"ecfdcbf875981eb0a48d60cbc1003e95e865da3fd3fd6dec53c35397b6308e43"} Nov 03 21:45:33.072606 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:33.072537 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-54674f497-2g5ct"] Nov 03 21:45:33.075840 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:45:33.075810 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00c924c1_ac0d_49c8_8610_3b837fbb8277.slice/crio-3eb641ca1e372ef33112a45ca00729446a14092b592b2c08fe59d58f33361bf4 WatchSource:0}: Error finding container 3eb641ca1e372ef33112a45ca00729446a14092b592b2c08fe59d58f33361bf4: Status 404 returned error can't find the container with id 3eb641ca1e372ef33112a45ca00729446a14092b592b2c08fe59d58f33361bf4 Nov 03 21:45:33.770165 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:33.770111 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jdbs4" event={"ID":"bfd8df86-d3d0-4576-afd0-fa742a6a3aad","Type":"ContainerStarted","Data":"50b3c526d3543eae9b1b46ce3a798c333a33dc9e2b07f0c7e23bb4eb04a762b9"} Nov 03 21:45:33.770554 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:33.770192 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jdbs4" event={"ID":"bfd8df86-d3d0-4576-afd0-fa742a6a3aad","Type":"ContainerStarted","Data":"868a0ad5fa19e4e821fa25c1387baa9caedc9af838abaa3b7da68abef9e6b7cc"} Nov 03 21:45:33.772714 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:33.772630 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" event={"ID":"00c924c1-ac0d-49c8-8610-3b837fbb8277","Type":"ContainerStarted","Data":"3eb641ca1e372ef33112a45ca00729446a14092b592b2c08fe59d58f33361bf4"} Nov 03 21:45:33.793770 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:33.793714 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-jdbs4" podStartSLOduration=2.799358516 podStartE2EDuration="3.793696065s" totalImagesPullingTime="925.670354ms" totalInitContainerRuntime="68.667195ms" isStatefulPod=true podCreationTimestamp="2026-11-03 21:45:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:45:33.792835109 +0000 UTC m=+50.102064124" watchObservedRunningTime="2026-11-03 21:45:33.793696065 +0000 UTC m=+50.102925080" Nov 03 21:45:34.778810 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:34.778776 2578 generic.go:356] "Generic (PLEG): container finished" podID="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" containerID="1d7e1a070f9e0542150676350d5dbd14e69951f115872bce24a88b74c9fb9a68" exitCode=0 Nov 03 21:45:34.779292 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:34.778876 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a2ccde5e-94a2-47ac-aff9-65248e4a6c92","Type":"ContainerDied","Data":"1d7e1a070f9e0542150676350d5dbd14e69951f115872bce24a88b74c9fb9a68"} Nov 03 21:45:35.152097 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:35.152066 2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-9cnqh"] Nov 03 21:45:35.154551 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:35.154536 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-9cnqh" Nov 03 21:45:35.164307 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:35.164279 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Nov 03 21:45:35.164307 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:35.164299 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-khpcw\"" Nov 03 21:45:35.165777 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:35.165751 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-9cnqh"] Nov 03 21:45:35.341854 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:35.341826 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a6cf0029-66be-4d86-ae8e-713d173a113e-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-9cnqh\" (UID: \"a6cf0029-66be-4d86-ae8e-713d173a113e\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-9cnqh" Nov 03 21:45:35.341969 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:35.341880 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfkkl\" (UniqueName: \"kubernetes.io/projected/a6cf0029-66be-4d86-ae8e-713d173a113e-kube-api-access-zfkkl\") pod \"monitoring-plugin-77f4f76594-9cnqh\" (UID: \"a6cf0029-66be-4d86-ae8e-713d173a113e\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-9cnqh" Nov 03 21:45:35.443207 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:35.443100 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a6cf0029-66be-4d86-ae8e-713d173a113e-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-9cnqh\" (UID: \"a6cf0029-66be-4d86-ae8e-713d173a113e\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-9cnqh" Nov 03 21:45:35.443339 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:35.443208 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfkkl\" (UniqueName: \"kubernetes.io/projected/a6cf0029-66be-4d86-ae8e-713d173a113e-kube-api-access-zfkkl\") pod \"monitoring-plugin-77f4f76594-9cnqh\" (UID: \"a6cf0029-66be-4d86-ae8e-713d173a113e\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-9cnqh" Nov 03 21:45:35.446219 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:35.446191 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a6cf0029-66be-4d86-ae8e-713d173a113e-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-9cnqh\" (UID: \"a6cf0029-66be-4d86-ae8e-713d173a113e\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-9cnqh" Nov 03 21:45:35.461654 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:35.461626 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfkkl\" (UniqueName: \"kubernetes.io/projected/a6cf0029-66be-4d86-ae8e-713d173a113e-kube-api-access-zfkkl\") pod \"monitoring-plugin-77f4f76594-9cnqh\" (UID: \"a6cf0029-66be-4d86-ae8e-713d173a113e\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-9cnqh" Nov 03 21:45:35.473341 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:35.473316 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-9cnqh" Nov 03 21:45:35.757053 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:35.757024 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-9cnqh"] Nov 03 21:45:35.759513 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:45:35.759485 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6cf0029_66be_4d86_ae8e_713d173a113e.slice/crio-3cb26baa6890fd84328a69c267cd625706f3fd3c705d645908c5d46f151828b3 WatchSource:0}: Error finding container 3cb26baa6890fd84328a69c267cd625706f3fd3c705d645908c5d46f151828b3: Status 404 returned error can't find the container with id 3cb26baa6890fd84328a69c267cd625706f3fd3c705d645908c5d46f151828b3 Nov 03 21:45:35.785203 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:35.785010 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" event={"ID":"00c924c1-ac0d-49c8-8610-3b837fbb8277","Type":"ContainerStarted","Data":"82eb44b83a256aaf322853703c039d80b60d0bf4ee1e3961842b6715dfb15303"} Nov 03 21:45:35.785203 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:35.785098 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" event={"ID":"00c924c1-ac0d-49c8-8610-3b837fbb8277","Type":"ContainerStarted","Data":"5d7efadc017441166915c1812083ba1345c21a5f5ac11b42302dc4b6efc842cd"} Nov 03 21:45:35.785203 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:35.785150 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" event={"ID":"00c924c1-ac0d-49c8-8610-3b837fbb8277","Type":"ContainerStarted","Data":"41b73c6e712f97ae039529a4c8be5099a281fa58d1a3b06d1f120909e662f8cc"} Nov 03 21:45:35.786401 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:35.786366 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-9cnqh" event={"ID":"a6cf0029-66be-4d86-ae8e-713d173a113e","Type":"ContainerStarted","Data":"3cb26baa6890fd84328a69c267cd625706f3fd3c705d645908c5d46f151828b3"} Nov 03 21:45:35.788340 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:35.788300 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86cf959768-tf4fd" event={"ID":"866ddb5a-c929-40c1-911d-ca40c5405723","Type":"ContainerStarted","Data":"777c53c0c6f5e7620a73772d2a60d00f6eb82ad24cbe6a2cb344b1bc9f4ebcde"} Nov 03 21:45:35.811450 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:35.811388 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-86cf959768-tf4fd" podStartSLOduration=1.7722728079999999 podStartE2EDuration="4.811372756s" totalImagesPullingTime="3.039099948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:45:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:45:35.809487923 +0000 UTC m=+52.118716953" watchObservedRunningTime="2026-11-03 21:45:35.811372756 +0000 UTC m=+52.120601770" Nov 03 21:45:36.672039 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.672004 2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 03 21:45:36.675140 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.675098 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.686894 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.686867 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Nov 03 21:45:36.688086 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.687689 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Nov 03 21:45:36.688086 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.687728 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Nov 03 21:45:36.688086 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.687787 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Nov 03 21:45:36.688086 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.687821 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Nov 03 21:45:36.688086 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.687857 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-52l5qoid38f49\"" Nov 03 21:45:36.688086 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.687876 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-6hj4n\"" Nov 03 21:45:36.688086 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.687897 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Nov 03 21:45:36.688086 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.687828 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Nov 03 21:45:36.688086 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.688008 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Nov 03 21:45:36.688086 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.688040 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Nov 03 21:45:36.688086 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.688058 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Nov 03 21:45:36.688611 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.688502 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Nov 03 21:45:36.688905 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.688849 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Nov 03 21:45:36.688978 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.688913 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Nov 03 21:45:36.693318 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.693292 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Nov 03 21:45:36.693418 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.693319 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Nov 03 21:45:36.696274 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.696253 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 03 21:45:36.796231 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.795979 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" event={"ID":"00c924c1-ac0d-49c8-8610-3b837fbb8277","Type":"ContainerStarted","Data":"5b3a3365aca252c086ebd78af11b117fda3afabc058aa19ac917eee00a873188"} Nov 03 21:45:36.796231 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.796070 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" event={"ID":"00c924c1-ac0d-49c8-8610-3b837fbb8277","Type":"ContainerStarted","Data":"707946beb12d1e31262e5faf9022aa65d3e1e084be4ac7d2f6e42072e50e310b"} Nov 03 21:45:36.796231 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.796116 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" event={"ID":"00c924c1-ac0d-49c8-8610-3b837fbb8277","Type":"ContainerStarted","Data":"55fb42af423654a63c28c1b762fca02e95dba38b0fc0c343ca3d73195581cae6"} Nov 03 21:45:36.796231 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.796177 2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:36.799864 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.799832 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a2ccde5e-94a2-47ac-aff9-65248e4a6c92","Type":"ContainerStarted","Data":"1ac717af4b9b79b11ecbbb6d351a9e05b4d7f1c903b2164fa001e3a33863d719"} Nov 03 21:45:36.799994 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.799907 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a2ccde5e-94a2-47ac-aff9-65248e4a6c92","Type":"ContainerStarted","Data":"ff688c729e043f4e44ad64cdf30845d30351c59caca4b5c995947f01364693aa"} Nov 03 21:45:36.800059 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.800009 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a2ccde5e-94a2-47ac-aff9-65248e4a6c92","Type":"ContainerStarted","Data":"b644bdeff48ac3f9a1a191846ea60b2dc37dbb46b3b0d14c78af59193858685e"} Nov 03 21:45:36.800059 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.800049 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a2ccde5e-94a2-47ac-aff9-65248e4a6c92","Type":"ContainerStarted","Data":"4c9079c6c502168773c81a19b51f0fc5c0f96e4788ddc40dbe93dc4e907bda46"} Nov 03 21:45:36.800177 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.800082 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a2ccde5e-94a2-47ac-aff9-65248e4a6c92","Type":"ContainerStarted","Data":"58b5804393fb6229f77781e0eb3ddcaaa735b2844861751aec958351c04fc520"} Nov 03 21:45:36.800177 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.800116 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a2ccde5e-94a2-47ac-aff9-65248e4a6c92","Type":"ContainerStarted","Data":"fac668a5bc5de0acff68159a1e9e2a066fec806a7736cd929c31307f799c04a3"} Nov 03 21:45:36.825853 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.825788 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" podStartSLOduration=2.055281302 podStartE2EDuration="4.825767758s" totalImagesPullingTime="2.770486456s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:45:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:45:36.823785677 +0000 UTC m=+53.133014729" watchObservedRunningTime="2026-11-03 21:45:36.825767758 +0000 UTC m=+53.134996774" Nov 03 21:45:36.859244 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.859195 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.859393 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.859277 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.859393 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.859346 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.859393 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.859373 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.859527 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.859414 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-config-out\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.859527 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.859479 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.859527 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.859521 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.859622 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.859555 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.859622 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.859575 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.859622 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.859590 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.859622 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.859609 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.859767 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.859634 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlgsq\" (UniqueName: \"kubernetes.io/projected/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-kube-api-access-mlgsq\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.859767 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.859654 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.859767 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.859696 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.859767 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.859719 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-config\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.859767 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.859733 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.859767 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.859749 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-web-config\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.859767 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.859764 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.859980 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.859782 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.859980 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.859802 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.886040 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.885977 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.417483615 podStartE2EDuration="5.885957866s" totalImagesPullingTime="2.452213531s" totalInitContainerRuntime="16.26072ms" isStatefulPod=false podCreationTimestamp="2026-11-03 21:45:31 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:45:36.884257453 +0000 UTC m=+53.193486470" watchObservedRunningTime="2026-11-03 21:45:36.885957866 +0000 UTC m=+53.195186885" Nov 03 21:45:36.960584 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.960503 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.960584 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.960549 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.960769 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.960594 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-config-out\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.960769 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.960616 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.960769 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.960666 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.960882 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.960772 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.960882 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.960799 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.960882 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.960819 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.960882 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.960840 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.961081 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.960916 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlgsq\" (UniqueName: \"kubernetes.io/projected/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-kube-api-access-mlgsq\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.961081 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.960967 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.961168 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.961088 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.961168 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.961144 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-config\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.961250 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.961169 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.961250 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.961195 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-web-config\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.961250 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.961219 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.961391 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.961262 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.961391 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.961291 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.961391 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.961335 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.961528 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.961425 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.961616 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.961593 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.963567 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.961964 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.963567 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.962031 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.963567 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.962668 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.963567 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.963565 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.965803 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.965577 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.966898 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.966876 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-config-out\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.967662 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.967618 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.968498 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.968466 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.969661 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.969617 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.969785 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.969765 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.969956 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.969934 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.970592 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.970568 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.970683 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.970651 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-config\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.970729 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.970710 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.971251 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.971227 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.971915 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.971888 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.972443 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.972421 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-web-config\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.975193 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.973752 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.977226 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.977202 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlgsq\" (UniqueName: \"kubernetes.io/projected/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-kube-api-access-mlgsq\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:36.998008 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:36.997972 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:37.296782 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:37.296709 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 03 21:45:37.299812 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:45:37.299781 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a5cbb44_3dbf_40a1_916b_71465d2dc2ea.slice/crio-12d4118e93b819d9eb3d110e90fce86db0b42341bf9196e7600928553ea1077a WatchSource:0}: Error finding container 12d4118e93b819d9eb3d110e90fce86db0b42341bf9196e7600928553ea1077a: Status 404 returned error can't find the container with id 12d4118e93b819d9eb3d110e90fce86db0b42341bf9196e7600928553ea1077a Nov 03 21:45:37.804597 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:37.804562 2578 generic.go:356] "Generic (PLEG): container finished" podID="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" containerID="14b9495cfb235a428a51ba26e6ccea26a7e4b5c67bbe3b4f43a6d924eff2decf" exitCode=0 Nov 03 21:45:37.805016 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:37.804643 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea","Type":"ContainerDied","Data":"14b9495cfb235a428a51ba26e6ccea26a7e4b5c67bbe3b4f43a6d924eff2decf"} Nov 03 21:45:37.805016 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:37.804739 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea","Type":"ContainerStarted","Data":"12d4118e93b819d9eb3d110e90fce86db0b42341bf9196e7600928553ea1077a"} Nov 03 21:45:37.806226 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:37.806204 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-9cnqh" event={"ID":"a6cf0029-66be-4d86-ae8e-713d173a113e","Type":"ContainerStarted","Data":"e2de832a8653a076be7948b7563de78ea29f099686cf83ebbb9c474724a930f5"} Nov 03 21:45:37.858722 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:37.858665 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-77f4f76594-9cnqh" podStartSLOduration=1.5034120340000001 podStartE2EDuration="2.858652001s" totalImagesPullingTime="1.355239967s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:45:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:45:37.857252065 +0000 UTC m=+54.166481103" watchObservedRunningTime="2026-11-03 21:45:37.858652001 +0000 UTC m=+54.167881016" Nov 03 21:45:38.696575 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:38.696548 2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:38.811626 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:38.811595 2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-9cnqh" Nov 03 21:45:38.817218 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:38.817192 2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-9cnqh" Nov 03 21:45:40.259816 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:40.259777 2578 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86cf959768-tf4fd"] Nov 03 21:45:40.757510 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:40.757480 2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-x2r5s" Nov 03 21:45:41.536005 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:41.535966 2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:41.825796 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:41.825707 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea","Type":"ContainerStarted","Data":"a049ac28ba7cf6a3af10022353c8a212963dfa04ff032c77d3a428dac1c83f0e"} Nov 03 21:45:41.825796 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:41.825780 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea","Type":"ContainerStarted","Data":"9d5d3e4daab369ff5f40ed2108677706b29a2250f185a7207a41b26939615efb"} Nov 03 21:45:41.825997 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:41.825827 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea","Type":"ContainerStarted","Data":"50fd1fee4f8ba5e31f1b78656c421ef48ede3afd8e4f85f7fe26540d536d502f"} Nov 03 21:45:41.825997 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:41.825862 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea","Type":"ContainerStarted","Data":"64d36f5b414bd0de01d0058f60c37b3051fe2f67b3b5834c066ae2d742c4d487"} Nov 03 21:45:41.825997 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:41.825896 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea","Type":"ContainerStarted","Data":"8fc4b9b4b217355d6d2c046bd20b54b62a6fa33031b4812bbe555ec7d6771f8d"} Nov 03 21:45:41.825997 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:41.825936 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea","Type":"ContainerStarted","Data":"1f9121440b907bd547107b875553716b36cf846bf8b11f652a22435d05b362ab"} Nov 03 21:45:41.862994 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:41.862939 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.874100041 podStartE2EDuration="5.862922011s" totalImagesPullingTime="2.973422568s" totalInitContainerRuntime="15.399402ms" isStatefulPod=false podCreationTimestamp="2026-11-03 21:45:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:45:41.861063789 +0000 UTC m=+58.170292846" watchObservedRunningTime="2026-11-03 21:45:41.862922011 +0000 UTC m=+58.172151025" Nov 03 21:45:41.998905 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:41.998875 2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:42.812413 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:42.812380 2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:48.979889 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:48.979851 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/94aaa3ac-c7fc-4f87-878e-8ef79127488b-original-pull-secret\") pod \"global-pull-secret-syncer-jq6hj\" (UID: \"94aaa3ac-c7fc-4f87-878e-8ef79127488b\") " pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:45:48.980319 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:48.979949 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb5f720e-f465-439f-ba32-4f595561f9f6-metrics-certs\") pod \"network-metrics-daemon-knt57\" (UID: \"eb5f720e-f465-439f-ba32-4f595561f9f6\") " pod="openshift-multus/network-metrics-daemon-knt57" Nov 03 21:45:48.984085 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:48.984062 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Nov 03 21:45:48.984203 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:48.984080 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Nov 03 21:45:48.992683 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:48.992659 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/94aaa3ac-c7fc-4f87-878e-8ef79127488b-original-pull-secret\") pod \"global-pull-secret-syncer-jq6hj\" (UID: \"94aaa3ac-c7fc-4f87-878e-8ef79127488b\") " pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:45:48.992781 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:48.992747 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb5f720e-f465-439f-ba32-4f595561f9f6-metrics-certs\") pod \"network-metrics-daemon-knt57\" (UID: \"eb5f720e-f465-439f-ba32-4f595561f9f6\") " pod="openshift-multus/network-metrics-daemon-knt57" Nov 03 21:45:49.080506 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:49.080469 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh4kw\" (UniqueName: \"kubernetes.io/projected/5d7e7316-1bd4-42cf-8519-544c1e2a4526-kube-api-access-zh4kw\") pod \"network-check-target-zrbsd\" (UID: \"5d7e7316-1bd4-42cf-8519-544c1e2a4526\") " pod="openshift-network-diagnostics/network-check-target-zrbsd" Nov 03 21:45:49.083600 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:49.083581 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Nov 03 21:45:49.094571 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:49.094542 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Nov 03 21:45:49.104993 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:49.104970 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zh4kw\" (UniqueName: \"kubernetes.io/projected/5d7e7316-1bd4-42cf-8519-544c1e2a4526-kube-api-access-zh4kw\") pod \"network-check-target-zrbsd\" (UID: \"5d7e7316-1bd4-42cf-8519-544c1e2a4526\") " pod="openshift-network-diagnostics/network-check-target-zrbsd" Nov 03 21:45:49.189046 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:49.189022 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-t98zr\"" Nov 03 21:45:49.196156 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:49.196137 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knt57" Nov 03 21:45:49.197997 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:49.197982 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-p72km\"" Nov 03 21:45:49.205044 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:49.205027 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-5xcnq\"" Nov 03 21:45:49.206165 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:49.206138 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrbsd" Nov 03 21:45:49.212978 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:49.212956 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:45:49.475234 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:49.475203 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-knt57"] Nov 03 21:45:49.482196 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:45:49.482169 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb5f720e_f465_439f_ba32_4f595561f9f6.slice/crio-ab4491eda7139fd31fdc7848d7f296f2c1f81b905d51c048f12a2ae3ea71da00 WatchSource:0}: Error finding container ab4491eda7139fd31fdc7848d7f296f2c1f81b905d51c048f12a2ae3ea71da00: Status 404 returned error can't find the container with id ab4491eda7139fd31fdc7848d7f296f2c1f81b905d51c048f12a2ae3ea71da00 Nov 03 21:45:49.488421 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:49.488392 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zrbsd"] Nov 03 21:45:49.492945 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:45:49.492920 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d7e7316_1bd4_42cf_8519_544c1e2a4526.slice/crio-43c10bc2bd2b94e42dbe0ea4ce382277116893404dc90d6a70e48102ec9fcbeb WatchSource:0}: Error finding container 43c10bc2bd2b94e42dbe0ea4ce382277116893404dc90d6a70e48102ec9fcbeb: Status 404 returned error can't find the container with id 43c10bc2bd2b94e42dbe0ea4ce382277116893404dc90d6a70e48102ec9fcbeb Nov 03 21:45:49.501790 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:49.501742 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jq6hj"] Nov 03 21:45:49.504391 ip-10-0-137-7 kubenswrapper[2578]: W1103 21:45:49.504364 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94aaa3ac_c7fc_4f87_878e_8ef79127488b.slice/crio-b251d8a015579cceb541952f1ebcb8e675394a3ad80016d966f745b49bbfb30c WatchSource:0}: Error finding container b251d8a015579cceb541952f1ebcb8e675394a3ad80016d966f745b49bbfb30c: Status 404 returned error can't find the container with id b251d8a015579cceb541952f1ebcb8e675394a3ad80016d966f745b49bbfb30c Nov 03 21:45:49.853285 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:49.853250 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zrbsd" event={"ID":"5d7e7316-1bd4-42cf-8519-544c1e2a4526","Type":"ContainerStarted","Data":"43c10bc2bd2b94e42dbe0ea4ce382277116893404dc90d6a70e48102ec9fcbeb"} Nov 03 21:45:49.854370 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:49.854344 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-knt57" event={"ID":"eb5f720e-f465-439f-ba32-4f595561f9f6","Type":"ContainerStarted","Data":"ab4491eda7139fd31fdc7848d7f296f2c1f81b905d51c048f12a2ae3ea71da00"} Nov 03 21:45:49.855602 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:49.855578 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jq6hj" event={"ID":"94aaa3ac-c7fc-4f87-878e-8ef79127488b","Type":"ContainerStarted","Data":"b251d8a015579cceb541952f1ebcb8e675394a3ad80016d966f745b49bbfb30c"} Nov 03 21:45:51.867529 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:51.866929 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-knt57" event={"ID":"eb5f720e-f465-439f-ba32-4f595561f9f6","Type":"ContainerStarted","Data":"b3ea9a0b572069593a208ed8d42d9cb901b8babdb2ca8e10340c1a93789f3471"} Nov 03 21:45:51.867529 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:51.867005 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-knt57" event={"ID":"eb5f720e-f465-439f-ba32-4f595561f9f6","Type":"ContainerStarted","Data":"a194772cc7f6c592d33f9f073e80176f45565ef683297695264ebe0204ac748b"} Nov 03 21:45:51.888422 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:51.888361 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-knt57" podStartSLOduration=66.60341134 podStartE2EDuration="1m7.888343167s" totalImagesPullingTime="1.284931827s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:44:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:45:51.885750282 +0000 UTC m=+68.194979296" watchObservedRunningTime="2026-11-03 21:45:51.888343167 +0000 UTC m=+68.197572187" Nov 03 21:45:53.876815 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:53.876779 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zrbsd" event={"ID":"5d7e7316-1bd4-42cf-8519-544c1e2a4526","Type":"ContainerStarted","Data":"f2e9cd127abd37190a6fd86ce9127d4f1f53c06f1aca2159701389dfe39933ef"} Nov 03 21:45:53.877329 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:53.876959 2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-zrbsd" Nov 03 21:45:53.894963 ip-10-0-137-7 kubenswrapper[2578]: I1103 21:45:53.894910 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-zrbsd" podStartSLOduration=66.478672353 podStartE2EDuration="1m9.894894591s" totalImagesPullingTime="3.416222238s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:44:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:45:53.893902971 +0000 UTC m=+70.203132011" watchObservedRunningTime="2026-11-03 21:45:53.894894591 +0000 UTC m=+70.204123605" Nov 03 21:45:54.579358 ip-10-0-137-7 systemd[1]: Stopping Kubernetes Kubelet... Nov 03 21:45:54.592540 ip-10-0-137-7 systemd[1]: kubelet.service: Deactivated successfully. Nov 03 21:45:54.592709 ip-10-0-137-7 systemd[1]: Stopped Kubernetes Kubelet. Nov 03 21:45:54.592889 ip-10-0-137-7 systemd[1]: kubelet.service: Consumed 4.582s CPU time. Nov 03 21:45:54.614730 ip-10-0-137-7 systemd[1]: Starting Kubernetes Kubelet... Nov 03 21:45:54.680361 ip-10-0-137-7 kubenswrapper[6524]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 03 21:45:54.680361 ip-10-0-137-7 kubenswrapper[6524]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Nov 03 21:45:54.680361 ip-10-0-137-7 kubenswrapper[6524]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 03 21:45:54.680361 ip-10-0-137-7 kubenswrapper[6524]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 03 21:45:54.682418 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682331 6524 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 03 21:45:54.682418 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682417 6524 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 03 21:45:54.682495 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682427 6524 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 03 21:45:54.682495 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682432 6524 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 03 21:45:54.682495 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682435 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 03 21:45:54.682495 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682439 6524 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 03 21:45:54.682495 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682442 6524 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 03 21:45:54.682495 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682445 6524 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 03 21:45:54.682495 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682447 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 03 21:45:54.682495 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682450 6524 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 03 21:45:54.682495 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682453 6524 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 03 21:45:54.682495 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682456 6524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 03 21:45:54.682495 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682459 6524 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 03 21:45:54.682495 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682461 6524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 03 21:45:54.682495 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682464 6524 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 03 21:45:54.682495 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682466 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 03 21:45:54.682495 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682470 6524 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 03 21:45:54.682495 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682474 6524 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 03 21:45:54.682495 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682478 6524 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 03 21:45:54.682495 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682480 6524 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 03 21:45:54.682495 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682483 6524 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 03 21:45:54.682994 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682486 6524 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 03 21:45:54.682994 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682489 6524 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 03 21:45:54.682994 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682491 6524 feature_gate.go:386] unrecognized feature gate: Example Nov 03 21:45:54.682994 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682494 6524 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 03 21:45:54.682994 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682497 6524 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 03 21:45:54.682994 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682500 6524 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 03 21:45:54.682994 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682503 6524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 03 21:45:54.682994 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682506 6524 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 03 21:45:54.682994 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682509 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 03 21:45:54.682994 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682511 6524 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 03 21:45:54.682994 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682514 6524 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 03 21:45:54.682994 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682517 6524 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 03 21:45:54.682994 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682519 6524 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 03 21:45:54.682994 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682524 6524 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 03 21:45:54.682994 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682529 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 03 21:45:54.682994 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682532 6524 feature_gate.go:386] unrecognized feature gate: EVPN Nov 03 21:45:54.682994 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682536 6524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 03 21:45:54.682994 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682539 6524 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 03 21:45:54.682994 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682542 6524 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 03 21:45:54.682994 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682544 6524 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 03 21:45:54.683538 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682547 6524 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 03 21:45:54.683538 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682549 6524 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 03 21:45:54.683538 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682552 6524 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 03 21:45:54.683538 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682555 6524 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 03 21:45:54.683538 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682557 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 03 21:45:54.683538 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682560 6524 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 03 21:45:54.683538 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682563 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 03 21:45:54.683538 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682565 6524 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 03 21:45:54.683538 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682568 6524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 03 21:45:54.683538 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682571 6524 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 03 21:45:54.683538 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682573 6524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 03 21:45:54.683538 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682576 6524 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 03 21:45:54.683538 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682579 6524 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 03 21:45:54.683538 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682581 6524 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 03 21:45:54.683538 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682585 6524 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 03 21:45:54.683538 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682587 6524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 03 21:45:54.683538 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682591 6524 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 03 21:45:54.683538 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682600 6524 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 03 21:45:54.683538 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682604 6524 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 03 21:45:54.684010 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682608 6524 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 03 21:45:54.684010 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682611 6524 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 03 21:45:54.684010 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682614 6524 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 03 21:45:54.684010 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682616 6524 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 03 21:45:54.684010 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682619 6524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 03 21:45:54.684010 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682623 6524 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 03 21:45:54.684010 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682626 6524 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 03 21:45:54.684010 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682629 6524 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 03 21:45:54.684010 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682631 6524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 03 21:45:54.684010 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682634 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 03 21:45:54.684010 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682636 6524 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 03 21:45:54.684010 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682639 6524 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 03 21:45:54.684010 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682641 6524 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 03 21:45:54.684010 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682644 6524 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 03 21:45:54.684010 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682646 6524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 03 21:45:54.684010 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682649 6524 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 03 21:45:54.684010 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682651 6524 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 03 21:45:54.684010 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682654 6524 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 03 21:45:54.684010 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682657 6524 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 03 21:45:54.684533 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682659 6524 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 03 21:45:54.684533 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682662 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 03 21:45:54.684533 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682665 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 03 21:45:54.684533 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682668 6524 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 03 21:45:54.684533 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682671 6524 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 03 21:45:54.684533 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682673 6524 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 03 21:45:54.684533 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682676 6524 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 03 21:45:54.684533 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682678 6524 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 03 21:45:54.684533 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682681 6524 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 03 21:45:54.684533 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682683 6524 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 03 21:45:54.684533 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682686 6524 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 03 21:45:54.684533 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682689 6524 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 03 21:45:54.684533 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682691 6524 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 03 21:45:54.684533 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682694 6524 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 03 21:45:54.684533 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682697 6524 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 03 21:45:54.684533 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682699 6524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 03 21:45:54.684533 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682702 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 03 21:45:54.684533 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682706 6524 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 03 21:45:54.684533 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682708 6524 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 03 21:45:54.684533 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682711 6524 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 03 21:45:54.685044 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682714 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 03 21:45:54.685044 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682716 6524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 03 21:45:54.685044 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682719 6524 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 03 21:45:54.685044 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682722 6524 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 03 21:45:54.685044 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682724 6524 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 03 21:45:54.685044 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682726 6524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 03 21:45:54.685044 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.682729 6524 feature_gate.go:386] unrecognized feature gate: Example2 Nov 03 21:45:54.685044 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683618 6524 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 03 21:45:54.685044 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683626 6524 feature_gate.go:386] unrecognized feature gate: Example2 Nov 03 21:45:54.685044 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683630 6524 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 03 21:45:54.685044 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683633 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 03 21:45:54.685044 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683636 6524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 03 21:45:54.685044 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683639 6524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 03 21:45:54.685044 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683642 6524 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 03 21:45:54.685044 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683645 6524 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 03 21:45:54.685044 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683648 6524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 03 21:45:54.685044 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683650 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 03 21:45:54.685044 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683653 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 03 21:45:54.685044 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683656 6524 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 03 21:45:54.685044 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683659 6524 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 03 21:45:54.685579 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683661 6524 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 03 21:45:54.685579 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683664 6524 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 03 21:45:54.685579 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683668 6524 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 03 21:45:54.685579 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683671 6524 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 03 21:45:54.685579 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683674 6524 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 03 21:45:54.685579 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683676 6524 feature_gate.go:386] unrecognized feature gate: Example Nov 03 21:45:54.685579 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683679 6524 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 03 21:45:54.685579 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683682 6524 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 03 21:45:54.685579 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683684 6524 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 03 21:45:54.685579 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683687 6524 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 03 21:45:54.685579 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683690 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 03 21:45:54.685579 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683693 6524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 03 21:45:54.685579 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683696 6524 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 03 21:45:54.685579 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683699 6524 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 03 21:45:54.685579 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683702 6524 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 03 21:45:54.685579 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683704 6524 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 03 21:45:54.685579 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683707 6524 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 03 21:45:54.685579 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683710 6524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 03 21:45:54.685579 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683712 6524 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 03 21:45:54.685579 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683715 6524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 03 21:45:54.686129 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683717 6524 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 03 21:45:54.686129 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683720 6524 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 03 21:45:54.686129 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683723 6524 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 03 21:45:54.686129 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683725 6524 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 03 21:45:54.686129 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683728 6524 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 03 21:45:54.686129 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683731 6524 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 03 21:45:54.686129 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683733 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 03 21:45:54.686129 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683736 6524 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 03 21:45:54.686129 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683739 6524 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 03 21:45:54.686129 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683741 6524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 03 21:45:54.686129 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683744 6524 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 03 21:45:54.686129 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683747 6524 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 03 21:45:54.686129 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683749 6524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 03 21:45:54.686129 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683753 6524 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 03 21:45:54.686129 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683757 6524 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 03 21:45:54.686129 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683761 6524 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 03 21:45:54.686129 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683764 6524 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 03 21:45:54.686129 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683767 6524 feature_gate.go:386] unrecognized feature gate: EVPN Nov 03 21:45:54.686129 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683771 6524 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 03 21:45:54.686617 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683775 6524 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 03 21:45:54.686617 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683778 6524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 03 21:45:54.686617 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683781 6524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 03 21:45:54.686617 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683783 6524 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 03 21:45:54.686617 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683786 6524 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 03 21:45:54.686617 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683788 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 03 21:45:54.686617 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683791 6524 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 03 21:45:54.686617 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683794 6524 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 03 21:45:54.686617 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683797 6524 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 03 21:45:54.686617 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683800 6524 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 03 21:45:54.686617 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683802 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 03 21:45:54.686617 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683805 6524 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 03 21:45:54.686617 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683807 6524 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 03 21:45:54.686617 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683810 6524 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 03 21:45:54.686617 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683812 6524 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 03 21:45:54.686617 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683815 6524 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 03 21:45:54.686617 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683818 6524 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 03 21:45:54.686617 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683820 6524 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 03 21:45:54.686617 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683823 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 03 21:45:54.686617 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683826 6524 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 03 21:45:54.687115 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683828 6524 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 03 21:45:54.687115 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683831 6524 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 03 21:45:54.687115 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683834 6524 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 03 21:45:54.687115 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683836 6524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 03 21:45:54.687115 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683839 6524 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 03 21:45:54.687115 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683841 6524 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 03 21:45:54.687115 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683844 6524 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 03 21:45:54.687115 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683847 6524 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 03 21:45:54.687115 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683850 6524 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 03 21:45:54.687115 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683852 6524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 03 21:45:54.687115 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683855 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 03 21:45:54.687115 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683858 6524 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 03 21:45:54.687115 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683860 6524 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 03 21:45:54.687115 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683863 6524 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 03 21:45:54.687115 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683865 6524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 03 21:45:54.687115 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683868 6524 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 03 21:45:54.687115 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683871 6524 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 03 21:45:54.687115 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683873 6524 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 03 21:45:54.687115 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683875 6524 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 03 21:45:54.687115 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683878 6524 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 03 21:45:54.687631 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683881 6524 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 03 21:45:54.687631 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683884 6524 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 03 21:45:54.687631 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683887 6524 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 03 21:45:54.687631 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683889 6524 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 03 21:45:54.687631 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683892 6524 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 03 21:45:54.687631 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683895 6524 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 03 21:45:54.687631 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683897 6524 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 03 21:45:54.687631 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683900 6524 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 03 21:45:54.687631 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683902 6524 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 03 21:45:54.687631 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683905 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 03 21:45:54.687631 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683907 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 03 21:45:54.687631 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683910 6524 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 03 21:45:54.687631 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683912 6524 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 03 21:45:54.687631 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.683915 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 03 21:45:54.687631 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684443 6524 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 03 21:45:54.687631 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684449 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 03 21:45:54.687631 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684452 6524 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 03 21:45:54.687631 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684456 6524 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 03 21:45:54.687631 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684458 6524 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 03 21:45:54.687631 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684461 6524 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 03 21:45:54.688140 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684464 6524 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 03 21:45:54.688140 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684467 6524 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 03 21:45:54.688140 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684469 6524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 03 21:45:54.688140 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684472 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 03 21:45:54.688140 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684475 6524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 03 21:45:54.688140 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684477 6524 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 03 21:45:54.688140 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684480 6524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 03 21:45:54.688140 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684485 6524 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 03 21:45:54.688140 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684489 6524 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 03 21:45:54.688140 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684493 6524 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 03 21:45:54.688140 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684496 6524 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 03 21:45:54.688140 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684499 6524 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 03 21:45:54.688140 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684502 6524 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 03 21:45:54.688140 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684505 6524 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 03 21:45:54.688140 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684508 6524 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 03 21:45:54.688140 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684510 6524 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 03 21:45:54.688140 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684513 6524 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 03 21:45:54.688140 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684516 6524 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 03 21:45:54.688140 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684519 6524 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 03 21:45:54.688623 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684521 6524 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 03 21:45:54.688623 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684524 6524 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 03 21:45:54.688623 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684526 6524 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 03 21:45:54.688623 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684528 6524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 03 21:45:54.688623 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684531 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 03 21:45:54.688623 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684534 6524 feature_gate.go:386] unrecognized feature gate: Example Nov 03 21:45:54.688623 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684536 6524 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 03 21:45:54.688623 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684539 6524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 03 21:45:54.688623 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684544 6524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 03 21:45:54.688623 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684547 6524 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 03 21:45:54.688623 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684550 6524 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 03 21:45:54.688623 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684552 6524 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 03 21:45:54.688623 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684556 6524 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 03 21:45:54.688623 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684561 6524 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 03 21:45:54.688623 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684564 6524 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 03 21:45:54.688623 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684568 6524 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 03 21:45:54.688623 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684571 6524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 03 21:45:54.688623 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684573 6524 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 03 21:45:54.688623 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684576 6524 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 03 21:45:54.689112 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684578 6524 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 03 21:45:54.689112 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684581 6524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 03 21:45:54.689112 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684584 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 03 21:45:54.689112 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684586 6524 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 03 21:45:54.689112 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684589 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 03 21:45:54.689112 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684591 6524 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 03 21:45:54.689112 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684595 6524 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 03 21:45:54.689112 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684597 6524 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 03 21:45:54.689112 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684600 6524 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 03 21:45:54.689112 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684603 6524 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 03 21:45:54.689112 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684605 6524 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 03 21:45:54.689112 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684608 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 03 21:45:54.689112 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684610 6524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 03 21:45:54.689112 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684613 6524 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 03 21:45:54.689112 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684615 6524 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 03 21:45:54.689112 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684618 6524 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 03 21:45:54.689112 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684620 6524 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 03 21:45:54.689112 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684623 6524 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 03 21:45:54.689112 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684625 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 03 21:45:54.689112 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684627 6524 feature_gate.go:386] unrecognized feature gate: Example2 Nov 03 21:45:54.689869 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684630 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 03 21:45:54.689869 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684634 6524 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 03 21:45:54.689869 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684637 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 03 21:45:54.689869 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684639 6524 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 03 21:45:54.689869 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684642 6524 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 03 21:45:54.689869 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684646 6524 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 03 21:45:54.689869 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684649 6524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 03 21:45:54.689869 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684651 6524 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 03 21:45:54.689869 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684654 6524 feature_gate.go:386] unrecognized feature gate: EVPN Nov 03 21:45:54.689869 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684657 6524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 03 21:45:54.689869 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684659 6524 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 03 21:45:54.689869 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684662 6524 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 03 21:45:54.689869 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684665 6524 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 03 21:45:54.689869 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684667 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 03 21:45:54.689869 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684670 6524 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 03 21:45:54.689869 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684672 6524 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 03 21:45:54.689869 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684675 6524 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 03 21:45:54.689869 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684677 6524 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 03 21:45:54.689869 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684681 6524 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 03 21:45:54.689869 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684685 6524 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 03 21:45:54.690543 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684688 6524 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 03 21:45:54.690543 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684690 6524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 03 21:45:54.690543 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684693 6524 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 03 21:45:54.690543 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684695 6524 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 03 21:45:54.690543 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684698 6524 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 03 21:45:54.690543 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684700 6524 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 03 21:45:54.690543 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684703 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 03 21:45:54.690543 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684705 6524 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 03 21:45:54.690543 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684708 6524 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 03 21:45:54.690543 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684710 6524 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 03 21:45:54.690543 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684713 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 03 21:45:54.690543 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684715 6524 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 03 21:45:54.690543 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684718 6524 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 03 21:45:54.690543 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684721 6524 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 03 21:45:54.690543 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684724 6524 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 03 21:45:54.690543 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684726 6524 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 03 21:45:54.690543 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684729 6524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 03 21:45:54.690543 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684732 6524 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 03 21:45:54.690543 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684735 6524 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 03 21:45:54.691022 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684738 6524 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 03 21:45:54.691022 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684740 6524 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 03 21:45:54.691022 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.684743 6524 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 03 21:45:54.691022 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.684837 6524 flags.go:64] FLAG: --address="0.0.0.0" Nov 03 21:45:54.691022 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.684847 6524 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Nov 03 21:45:54.691022 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.684857 6524 flags.go:64] FLAG: --anonymous-auth="true" Nov 03 21:45:54.691022 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.684864 6524 flags.go:64] FLAG: --application-metrics-count-limit="100" Nov 03 21:45:54.691022 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.684870 6524 flags.go:64] FLAG: --authentication-token-webhook="false" Nov 03 21:45:54.691022 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.684875 6524 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Nov 03 21:45:54.691022 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.684880 6524 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Nov 03 21:45:54.691022 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.684884 6524 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Nov 03 21:45:54.691022 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.684887 6524 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Nov 03 21:45:54.691022 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.684890 6524 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Nov 03 21:45:54.691022 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.684895 6524 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Nov 03 21:45:54.691022 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.684899 6524 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Nov 03 21:45:54.691022 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.684902 6524 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Nov 03 21:45:54.691022 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.684906 6524 flags.go:64] FLAG: --cgroup-root="" Nov 03 21:45:54.691022 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.684909 6524 flags.go:64] FLAG: --cgroups-per-qos="true" Nov 03 21:45:54.691022 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.684912 6524 flags.go:64] FLAG: --client-ca-file="" Nov 03 21:45:54.691022 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.684915 6524 flags.go:64] FLAG: --cloud-provider="external" Nov 03 21:45:54.691022 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.684918 6524 flags.go:64] FLAG: --cluster-dns="[]" Nov 03 21:45:54.691022 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.684922 6524 flags.go:64] FLAG: --cluster-domain="" Nov 03 21:45:54.691022 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.684925 6524 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Nov 03 21:45:54.691022 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.684928 6524 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Nov 03 21:45:54.691644 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.684932 6524 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Nov 03 21:45:54.691644 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.684935 6524 flags.go:64] FLAG: --container-log-max-files="5" Nov 03 21:45:54.691644 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.684941 6524 flags.go:64] FLAG: --container-log-max-size="10Mi" Nov 03 21:45:54.691644 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.684944 6524 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Nov 03 21:45:54.691644 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.684948 6524 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Nov 03 21:45:54.691644 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.684951 6524 flags.go:64] FLAG: --containerd-namespace="k8s.io" Nov 03 21:45:54.691644 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.684955 6524 flags.go:64] FLAG: --contention-profiling="false" Nov 03 21:45:54.691644 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.684958 6524 flags.go:64] FLAG: --cpu-cfs-quota="true" Nov 03 21:45:54.691644 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.684962 6524 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Nov 03 21:45:54.691644 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.684965 6524 flags.go:64] FLAG: --cpu-manager-policy="none" Nov 03 21:45:54.691644 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.684968 6524 flags.go:64] FLAG: --cpu-manager-policy-options="" Nov 03 21:45:54.691644 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.684972 6524 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Nov 03 21:45:54.691644 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.684976 6524 flags.go:64] FLAG: --enable-controller-attach-detach="true" Nov 03 21:45:54.691644 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.684979 6524 flags.go:64] FLAG: --enable-debugging-handlers="true" Nov 03 21:45:54.691644 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.684982 6524 flags.go:64] FLAG: --enable-load-reader="false" Nov 03 21:45:54.691644 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.684985 6524 flags.go:64] FLAG: --enable-server="true" Nov 03 21:45:54.691644 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.684988 6524 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Nov 03 21:45:54.691644 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.684992 6524 flags.go:64] FLAG: --event-burst="100" Nov 03 21:45:54.691644 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.684995 6524 flags.go:64] FLAG: --event-qps="50" Nov 03 21:45:54.691644 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.684998 6524 flags.go:64] FLAG: --event-storage-age-limit="default=0" Nov 03 21:45:54.691644 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685002 6524 flags.go:64] FLAG: --event-storage-event-limit="default=0" Nov 03 21:45:54.691644 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685005 6524 flags.go:64] FLAG: --eviction-hard="" Nov 03 21:45:54.691644 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685010 6524 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Nov 03 21:45:54.691644 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685013 6524 flags.go:64] FLAG: --eviction-minimum-reclaim="" Nov 03 21:45:54.692329 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685016 6524 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Nov 03 21:45:54.692329 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685019 6524 flags.go:64] FLAG: --eviction-soft="" Nov 03 21:45:54.692329 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685023 6524 flags.go:64] FLAG: --eviction-soft-grace-period="" Nov 03 21:45:54.692329 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685025 6524 flags.go:64] FLAG: --exit-on-lock-contention="false" Nov 03 21:45:54.692329 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685029 6524 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Nov 03 21:45:54.692329 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685032 6524 flags.go:64] FLAG: --experimental-mounter-path="" Nov 03 21:45:54.692329 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685035 6524 flags.go:64] FLAG: --fail-cgroupv1="true" Nov 03 21:45:54.692329 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685038 6524 flags.go:64] FLAG: --fail-swap-on="true" Nov 03 21:45:54.692329 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685041 6524 flags.go:64] FLAG: --feature-gates="" Nov 03 21:45:54.692329 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685045 6524 flags.go:64] FLAG: --file-check-frequency="20s" Nov 03 21:45:54.692329 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685050 6524 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Nov 03 21:45:54.692329 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685053 6524 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Nov 03 21:45:54.692329 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685057 6524 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Nov 03 21:45:54.692329 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685060 6524 flags.go:64] FLAG: --healthz-port="10248" Nov 03 21:45:54.692329 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685065 6524 flags.go:64] FLAG: --help="false" Nov 03 21:45:54.692329 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685068 6524 flags.go:64] FLAG: --hostname-override="ip-10-0-137-7.us-east-2.compute.internal" Nov 03 21:45:54.692329 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685072 6524 flags.go:64] FLAG: --housekeeping-interval="10s" Nov 03 21:45:54.692329 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685075 6524 flags.go:64] FLAG: --http-check-frequency="20s" Nov 03 21:45:54.692329 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685078 6524 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Nov 03 21:45:54.692329 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685081 6524 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Nov 03 21:45:54.692329 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685085 6524 flags.go:64] FLAG: --image-gc-high-threshold="85" Nov 03 21:45:54.692329 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685087 6524 flags.go:64] FLAG: --image-gc-low-threshold="80" Nov 03 21:45:54.692329 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685090 6524 flags.go:64] FLAG: --image-service-endpoint="" Nov 03 21:45:54.692915 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685093 6524 flags.go:64] FLAG: --kernel-memcg-notification="false" Nov 03 21:45:54.692915 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685097 6524 flags.go:64] FLAG: --kube-api-burst="100" Nov 03 21:45:54.692915 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685100 6524 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Nov 03 21:45:54.692915 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685103 6524 flags.go:64] FLAG: --kube-api-qps="50" Nov 03 21:45:54.692915 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685106 6524 flags.go:64] FLAG: --kube-reserved="" Nov 03 21:45:54.692915 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685109 6524 flags.go:64] FLAG: --kube-reserved-cgroup="" Nov 03 21:45:54.692915 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685112 6524 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Nov 03 21:45:54.692915 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685115 6524 flags.go:64] FLAG: --kubelet-cgroups="" Nov 03 21:45:54.692915 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685137 6524 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Nov 03 21:45:54.692915 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685141 6524 flags.go:64] FLAG: --lock-file="" Nov 03 21:45:54.692915 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685144 6524 flags.go:64] FLAG: --log-cadvisor-usage="false" Nov 03 21:45:54.692915 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685147 6524 flags.go:64] FLAG: --log-flush-frequency="5s" Nov 03 21:45:54.692915 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685151 6524 flags.go:64] FLAG: --log-json-info-buffer-size="0" Nov 03 21:45:54.692915 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685156 6524 flags.go:64] FLAG: --log-json-split-stream="false" Nov 03 21:45:54.692915 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685159 6524 flags.go:64] FLAG: --log-text-info-buffer-size="0" Nov 03 21:45:54.692915 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685162 6524 flags.go:64] FLAG: --log-text-split-stream="false" Nov 03 21:45:54.692915 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685166 6524 flags.go:64] FLAG: --logging-format="text" Nov 03 21:45:54.692915 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685169 6524 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Nov 03 21:45:54.692915 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685172 6524 flags.go:64] FLAG: --make-iptables-util-chains="true" Nov 03 21:45:54.692915 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685176 6524 flags.go:64] FLAG: --manifest-url="" Nov 03 21:45:54.692915 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685179 6524 flags.go:64] FLAG: --manifest-url-header="" Nov 03 21:45:54.692915 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685184 6524 flags.go:64] FLAG: --max-housekeeping-interval="15s" Nov 03 21:45:54.692915 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685188 6524 flags.go:64] FLAG: --max-open-files="1000000" Nov 03 21:45:54.692915 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685194 6524 flags.go:64] FLAG: --max-pods="110" Nov 03 21:45:54.692915 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685197 6524 flags.go:64] FLAG: --maximum-dead-containers="-1" Nov 03 21:45:54.693539 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685200 6524 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Nov 03 21:45:54.693539 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685203 6524 flags.go:64] FLAG: --memory-manager-policy="None" Nov 03 21:45:54.693539 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685206 6524 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Nov 03 21:45:54.693539 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685209 6524 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Nov 03 21:45:54.693539 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685212 6524 flags.go:64] FLAG: --node-ip="0.0.0.0" Nov 03 21:45:54.693539 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685215 6524 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Nov 03 21:45:54.693539 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685223 6524 flags.go:64] FLAG: --node-status-max-images="50" Nov 03 21:45:54.693539 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685226 6524 flags.go:64] FLAG: --node-status-update-frequency="10s" Nov 03 21:45:54.693539 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685229 6524 flags.go:64] FLAG: --oom-score-adj="-999" Nov 03 21:45:54.693539 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685232 6524 flags.go:64] FLAG: --pod-cidr="" Nov 03 21:45:54.693539 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685235 6524 flags.go:64] FLAG: --pod-manifest-path="" Nov 03 21:45:54.693539 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685239 6524 flags.go:64] FLAG: --pod-max-pids="-1" Nov 03 21:45:54.693539 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685242 6524 flags.go:64] FLAG: --pods-per-core="0" Nov 03 21:45:54.693539 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685245 6524 flags.go:64] FLAG: --port="10250" Nov 03 21:45:54.693539 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685248 6524 flags.go:64] FLAG: --protect-kernel-defaults="false" Nov 03 21:45:54.693539 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685251 6524 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-0aae6af7f38c14baa" Nov 03 21:45:54.693539 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685255 6524 flags.go:64] FLAG: --qos-reserved="" Nov 03 21:45:54.693539 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685259 6524 flags.go:64] FLAG: --read-only-port="10255" Nov 03 21:45:54.693539 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685262 6524 flags.go:64] FLAG: --register-node="true" Nov 03 21:45:54.693539 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685265 6524 flags.go:64] FLAG: --register-with-taints="" Nov 03 21:45:54.693539 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685273 6524 flags.go:64] FLAG: --registry-burst="10" Nov 03 21:45:54.693539 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685276 6524 flags.go:64] FLAG: --registry-qps="5" Nov 03 21:45:54.693539 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685279 6524 flags.go:64] FLAG: --reserved-cpus="" Nov 03 21:45:54.693539 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685282 6524 flags.go:64] FLAG: --reserved-memory="" Nov 03 21:45:54.693539 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685286 6524 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Nov 03 21:45:54.693539 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685290 6524 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Nov 03 21:45:54.694203 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685294 6524 flags.go:64] FLAG: --rotate-certificates="false" Nov 03 21:45:54.694203 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685297 6524 flags.go:64] FLAG: --rotate-server-certificates="false" Nov 03 21:45:54.694203 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685300 6524 flags.go:64] FLAG: --runonce="false" Nov 03 21:45:54.694203 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685303 6524 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Nov 03 21:45:54.694203 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685307 6524 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Nov 03 21:45:54.694203 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685311 6524 flags.go:64] FLAG: --seccomp-default="false" Nov 03 21:45:54.694203 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685313 6524 flags.go:64] FLAG: --serialize-image-pulls="true" Nov 03 21:45:54.694203 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685317 6524 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Nov 03 21:45:54.694203 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685320 6524 flags.go:64] FLAG: --storage-driver-db="cadvisor" Nov 03 21:45:54.694203 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685324 6524 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Nov 03 21:45:54.694203 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685327 6524 flags.go:64] FLAG: --storage-driver-password="root" Nov 03 21:45:54.694203 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685331 6524 flags.go:64] FLAG: --storage-driver-secure="false" Nov 03 21:45:54.694203 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685334 6524 flags.go:64] FLAG: --storage-driver-table="stats" Nov 03 21:45:54.694203 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685337 6524 flags.go:64] FLAG: --storage-driver-user="root" Nov 03 21:45:54.694203 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685339 6524 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Nov 03 21:45:54.694203 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685342 6524 flags.go:64] FLAG: --sync-frequency="1m0s" Nov 03 21:45:54.694203 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685345 6524 flags.go:64] FLAG: --system-cgroups="" Nov 03 21:45:54.694203 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685349 6524 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Nov 03 21:45:54.694203 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685354 6524 flags.go:64] FLAG: --system-reserved-cgroup="" Nov 03 21:45:54.694203 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685357 6524 flags.go:64] FLAG: --tls-cert-file="" Nov 03 21:45:54.694203 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685360 6524 flags.go:64] FLAG: --tls-cipher-suites="[]" Nov 03 21:45:54.694203 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685364 6524 flags.go:64] FLAG: --tls-min-version="" Nov 03 21:45:54.694203 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685367 6524 flags.go:64] FLAG: --tls-private-key-file="" Nov 03 21:45:54.694203 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685371 6524 flags.go:64] FLAG: --topology-manager-policy="none" Nov 03 21:45:54.694203 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685375 6524 flags.go:64] FLAG: --topology-manager-policy-options="" Nov 03 21:45:54.694848 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685378 6524 flags.go:64] FLAG: --topology-manager-scope="container" Nov 03 21:45:54.694848 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685381 6524 flags.go:64] FLAG: --v="2" Nov 03 21:45:54.694848 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685386 6524 flags.go:64] FLAG: --version="false" Nov 03 21:45:54.694848 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685390 6524 flags.go:64] FLAG: --vmodule="" Nov 03 21:45:54.694848 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685394 6524 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Nov 03 21:45:54.694848 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685398 6524 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Nov 03 21:45:54.694848 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685555 6524 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 03 21:45:54.694848 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685560 6524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 03 21:45:54.694848 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685563 6524 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 03 21:45:54.694848 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685566 6524 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 03 21:45:54.694848 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685569 6524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 03 21:45:54.694848 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685573 6524 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 03 21:45:54.694848 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685576 6524 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 03 21:45:54.694848 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685580 6524 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 03 21:45:54.694848 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685584 6524 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 03 21:45:54.694848 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685588 6524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 03 21:45:54.694848 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685591 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 03 21:45:54.694848 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685594 6524 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 03 21:45:54.694848 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685597 6524 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 03 21:45:54.694848 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685600 6524 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 03 21:45:54.694848 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685603 6524 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 03 21:45:54.695383 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685605 6524 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 03 21:45:54.695383 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685608 6524 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 03 21:45:54.695383 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685611 6524 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 03 21:45:54.695383 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685613 6524 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 03 21:45:54.695383 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685616 6524 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 03 21:45:54.695383 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685619 6524 feature_gate.go:386] unrecognized feature gate: Example Nov 03 21:45:54.695383 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685621 6524 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 03 21:45:54.695383 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685624 6524 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 03 21:45:54.695383 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685626 6524 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 03 21:45:54.695383 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685629 6524 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 03 21:45:54.695383 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685633 6524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 03 21:45:54.695383 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685635 6524 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 03 21:45:54.695383 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685639 6524 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 03 21:45:54.695383 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685642 6524 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 03 21:45:54.695383 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685645 6524 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 03 21:45:54.695383 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685647 6524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 03 21:45:54.695383 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685650 6524 feature_gate.go:386] unrecognized feature gate: Example2 Nov 03 21:45:54.695383 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685653 6524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 03 21:45:54.695383 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685658 6524 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 03 21:45:54.695383 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685661 6524 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 03 21:45:54.695896 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685664 6524 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 03 21:45:54.695896 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685666 6524 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 03 21:45:54.695896 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685669 6524 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 03 21:45:54.695896 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685672 6524 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 03 21:45:54.695896 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685675 6524 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 03 21:45:54.695896 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685677 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 03 21:45:54.695896 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685680 6524 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 03 21:45:54.695896 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685682 6524 feature_gate.go:386] unrecognized feature gate: EVPN Nov 03 21:45:54.695896 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685685 6524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 03 21:45:54.695896 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685688 6524 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 03 21:45:54.695896 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685690 6524 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 03 21:45:54.695896 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685693 6524 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 03 21:45:54.695896 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685695 6524 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 03 21:45:54.695896 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685698 6524 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 03 21:45:54.695896 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685700 6524 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 03 21:45:54.695896 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685703 6524 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 03 21:45:54.695896 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685705 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 03 21:45:54.695896 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685708 6524 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 03 21:45:54.695896 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685711 6524 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 03 21:45:54.696689 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685713 6524 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 03 21:45:54.696689 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685715 6524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 03 21:45:54.696689 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685718 6524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 03 21:45:54.696689 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685721 6524 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 03 21:45:54.696689 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685724 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 03 21:45:54.696689 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685727 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 03 21:45:54.696689 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685729 6524 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 03 21:45:54.696689 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685732 6524 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 03 21:45:54.696689 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685735 6524 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 03 21:45:54.696689 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685737 6524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 03 21:45:54.696689 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685739 6524 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 03 21:45:54.696689 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685742 6524 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 03 21:45:54.696689 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685744 6524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 03 21:45:54.696689 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685747 6524 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 03 21:45:54.696689 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685750 6524 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 03 21:45:54.696689 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685753 6524 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 03 21:45:54.696689 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685756 6524 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 03 21:45:54.696689 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685759 6524 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 03 21:45:54.696689 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685761 6524 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 03 21:45:54.696689 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685764 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 03 21:45:54.697501 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685767 6524 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 03 21:45:54.697501 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685770 6524 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 03 21:45:54.697501 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685773 6524 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 03 21:45:54.697501 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685775 6524 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 03 21:45:54.697501 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685778 6524 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 03 21:45:54.697501 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685780 6524 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 03 21:45:54.697501 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685783 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 03 21:45:54.697501 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685785 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 03 21:45:54.697501 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685788 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 03 21:45:54.697501 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685790 6524 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 03 21:45:54.697501 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685793 6524 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 03 21:45:54.697501 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685796 6524 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 03 21:45:54.697501 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685798 6524 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 03 21:45:54.697501 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685801 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 03 21:45:54.697501 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685803 6524 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 03 21:45:54.697501 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685806 6524 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 03 21:45:54.697501 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685809 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 03 21:45:54.697501 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685812 6524 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 03 21:45:54.697501 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685815 6524 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 03 21:45:54.698233 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685817 6524 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 03 21:45:54.698233 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685820 6524 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 03 21:45:54.698233 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685822 6524 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 03 21:45:54.698233 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685825 6524 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 03 21:45:54.698233 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685828 6524 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 03 21:45:54.698233 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685830 6524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 03 21:45:54.698233 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685833 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 03 21:45:54.698233 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685835 6524 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 03 21:45:54.698233 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685838 6524 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 03 21:45:54.698233 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685841 6524 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 03 21:45:54.698233 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685843 6524 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 03 21:45:54.698233 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685846 6524 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 03 21:45:54.698233 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685849 6524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 03 21:45:54.698233 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.685875 6524 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 03 21:45:54.698233 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685921 6524 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 03 21:45:54.698233 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685925 6524 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 03 21:45:54.698974 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685928 6524 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 03 21:45:54.698974 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685931 6524 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 03 21:45:54.698974 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685934 6524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 03 21:45:54.698974 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685937 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 03 21:45:54.698974 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685939 6524 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 03 21:45:54.698974 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685942 6524 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 03 21:45:54.698974 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685944 6524 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 03 21:45:54.698974 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685947 6524 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 03 21:45:54.698974 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685949 6524 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 03 21:45:54.698974 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685952 6524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 03 21:45:54.698974 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685954 6524 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 03 21:45:54.698974 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685957 6524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 03 21:45:54.698974 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685959 6524 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 03 21:45:54.698974 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685962 6524 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 03 21:45:54.698974 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685965 6524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 03 21:45:54.698974 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685968 6524 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 03 21:45:54.698974 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685971 6524 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 03 21:45:54.698974 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685974 6524 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 03 21:45:54.698974 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685978 6524 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 03 21:45:54.699742 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685982 6524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 03 21:45:54.699742 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685984 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 03 21:45:54.699742 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685987 6524 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 03 21:45:54.699742 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685991 6524 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 03 21:45:54.699742 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685994 6524 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 03 21:45:54.699742 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685996 6524 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 03 21:45:54.699742 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.685999 6524 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 03 21:45:54.699742 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686001 6524 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 03 21:45:54.699742 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686004 6524 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 03 21:45:54.699742 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686007 6524 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 03 21:45:54.699742 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686009 6524 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 03 21:45:54.699742 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686012 6524 feature_gate.go:386] unrecognized feature gate: Example Nov 03 21:45:54.699742 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686014 6524 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 03 21:45:54.699742 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686017 6524 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 03 21:45:54.699742 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686019 6524 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 03 21:45:54.699742 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686022 6524 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 03 21:45:54.699742 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686024 6524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 03 21:45:54.699742 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686027 6524 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 03 21:45:54.699742 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686029 6524 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 03 21:45:54.699742 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686032 6524 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 03 21:45:54.700401 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686035 6524 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 03 21:45:54.700401 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686037 6524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 03 21:45:54.700401 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686040 6524 feature_gate.go:386] unrecognized feature gate: Example2 Nov 03 21:45:54.700401 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686043 6524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 03 21:45:54.700401 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686047 6524 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 03 21:45:54.700401 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686051 6524 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 03 21:45:54.700401 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686054 6524 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 03 21:45:54.700401 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686058 6524 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 03 21:45:54.700401 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686060 6524 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 03 21:45:54.700401 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686064 6524 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 03 21:45:54.700401 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686066 6524 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 03 21:45:54.700401 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686069 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 03 21:45:54.700401 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686072 6524 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 03 21:45:54.700401 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686074 6524 feature_gate.go:386] unrecognized feature gate: EVPN Nov 03 21:45:54.700401 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686076 6524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 03 21:45:54.700401 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686080 6524 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 03 21:45:54.700401 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686083 6524 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 03 21:45:54.700401 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686086 6524 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 03 21:45:54.700401 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686088 6524 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 03 21:45:54.700931 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686091 6524 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 03 21:45:54.700931 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686093 6524 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 03 21:45:54.700931 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686096 6524 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 03 21:45:54.700931 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686099 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 03 21:45:54.700931 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686101 6524 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 03 21:45:54.700931 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686104 6524 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 03 21:45:54.700931 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686107 6524 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 03 21:45:54.700931 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686109 6524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 03 21:45:54.700931 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686112 6524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 03 21:45:54.700931 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686114 6524 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 03 21:45:54.700931 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686118 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 03 21:45:54.700931 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686133 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 03 21:45:54.700931 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686135 6524 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 03 21:45:54.700931 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686138 6524 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 03 21:45:54.700931 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686141 6524 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 03 21:45:54.700931 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686144 6524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 03 21:45:54.700931 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686147 6524 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 03 21:45:54.700931 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686149 6524 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 03 21:45:54.700931 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686152 6524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 03 21:45:54.701426 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686155 6524 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 03 21:45:54.701426 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686159 6524 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 03 21:45:54.701426 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686162 6524 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 03 21:45:54.701426 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686164 6524 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 03 21:45:54.701426 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686168 6524 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 03 21:45:54.701426 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686171 6524 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 03 21:45:54.701426 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686173 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 03 21:45:54.701426 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686176 6524 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 03 21:45:54.701426 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686179 6524 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 03 21:45:54.701426 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686183 6524 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 03 21:45:54.701426 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686185 6524 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 03 21:45:54.701426 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686188 6524 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 03 21:45:54.701426 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686191 6524 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 03 21:45:54.701426 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686193 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 03 21:45:54.701426 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686196 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 03 21:45:54.701426 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686199 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 03 21:45:54.701426 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686201 6524 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 03 21:45:54.701426 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686205 6524 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 03 21:45:54.701426 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686207 6524 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 03 21:45:54.701900 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686210 6524 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 03 21:45:54.701900 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686213 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 03 21:45:54.701900 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686215 6524 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 03 21:45:54.701900 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686218 6524 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 03 21:45:54.701900 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686220 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 03 21:45:54.701900 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686223 6524 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 03 21:45:54.701900 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686226 6524 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 03 21:45:54.701900 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.686229 6524 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 03 21:45:54.701900 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.691549 6524 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Nov 03 21:45:54.701900 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.691573 6524 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Nov 03 21:45:54.701900 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.691590 6524 watchdog_linux.go:95] "Systemd watchdog is not enabled" Nov 03 21:45:54.701900 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.691595 6524 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Nov 03 21:45:54.701900 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691659 6524 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 03 21:45:54.701900 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691664 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 03 21:45:54.701900 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691669 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 03 21:45:54.701900 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691672 6524 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 03 21:45:54.701900 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691675 6524 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 03 21:45:54.701900 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691678 6524 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 03 21:45:54.701900 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691681 6524 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 03 21:45:54.702386 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691683 6524 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 03 21:45:54.702386 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691686 6524 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 03 21:45:54.702386 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691689 6524 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 03 21:45:54.702386 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691692 6524 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 03 21:45:54.702386 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691695 6524 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 03 21:45:54.702386 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691699 6524 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 03 21:45:54.702386 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691703 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 03 21:45:54.702386 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691707 6524 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 03 21:45:54.702386 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691710 6524 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 03 21:45:54.702386 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691718 6524 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 03 21:45:54.702386 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691722 6524 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 03 21:45:54.702386 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691726 6524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 03 21:45:54.702386 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691729 6524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 03 21:45:54.702386 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691731 6524 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 03 21:45:54.702386 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691734 6524 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 03 21:45:54.702386 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691737 6524 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 03 21:45:54.702386 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691740 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 03 21:45:54.702386 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691743 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 03 21:45:54.702386 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691745 6524 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 03 21:45:54.702861 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691748 6524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 03 21:45:54.702861 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691751 6524 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 03 21:45:54.702861 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691754 6524 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 03 21:45:54.702861 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691757 6524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 03 21:45:54.702861 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691760 6524 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 03 21:45:54.702861 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691762 6524 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 03 21:45:54.702861 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691765 6524 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 03 21:45:54.702861 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691768 6524 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 03 21:45:54.702861 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691770 6524 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 03 21:45:54.702861 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691773 6524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 03 21:45:54.702861 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691775 6524 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 03 21:45:54.702861 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691778 6524 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 03 21:45:54.702861 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691780 6524 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 03 21:45:54.702861 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691783 6524 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 03 21:45:54.702861 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691786 6524 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 03 21:45:54.702861 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691788 6524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 03 21:45:54.702861 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691791 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 03 21:45:54.702861 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691793 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 03 21:45:54.702861 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691796 6524 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 03 21:45:54.702861 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691798 6524 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 03 21:45:54.703590 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691801 6524 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 03 21:45:54.703590 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691804 6524 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 03 21:45:54.703590 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691807 6524 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 03 21:45:54.703590 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691810 6524 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 03 21:45:54.703590 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691812 6524 feature_gate.go:386] unrecognized feature gate: EVPN Nov 03 21:45:54.703590 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691815 6524 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 03 21:45:54.703590 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691818 6524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 03 21:45:54.703590 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691820 6524 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 03 21:45:54.703590 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691823 6524 feature_gate.go:386] unrecognized feature gate: Example2 Nov 03 21:45:54.703590 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691825 6524 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 03 21:45:54.703590 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691829 6524 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 03 21:45:54.703590 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691831 6524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 03 21:45:54.703590 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691834 6524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 03 21:45:54.703590 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691836 6524 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 03 21:45:54.703590 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691839 6524 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 03 21:45:54.703590 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691843 6524 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 03 21:45:54.703590 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691847 6524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 03 21:45:54.703590 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691850 6524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 03 21:45:54.703590 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691853 6524 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 03 21:45:54.703590 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691855 6524 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 03 21:45:54.704092 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691858 6524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 03 21:45:54.704092 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691861 6524 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 03 21:45:54.704092 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691863 6524 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 03 21:45:54.704092 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691866 6524 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 03 21:45:54.704092 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691868 6524 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 03 21:45:54.704092 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691871 6524 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 03 21:45:54.704092 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691873 6524 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 03 21:45:54.704092 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691876 6524 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 03 21:45:54.704092 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691879 6524 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 03 21:45:54.704092 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691881 6524 feature_gate.go:386] unrecognized feature gate: Example Nov 03 21:45:54.704092 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691884 6524 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 03 21:45:54.704092 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691887 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 03 21:45:54.704092 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691889 6524 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 03 21:45:54.704092 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691893 6524 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 03 21:45:54.704092 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691896 6524 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 03 21:45:54.704092 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691899 6524 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 03 21:45:54.704092 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691901 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 03 21:45:54.704092 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691904 6524 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 03 21:45:54.704092 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691906 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 03 21:45:54.704614 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691909 6524 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 03 21:45:54.704614 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691912 6524 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 03 21:45:54.704614 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691914 6524 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 03 21:45:54.704614 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691916 6524 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 03 21:45:54.704614 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691919 6524 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 03 21:45:54.704614 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691921 6524 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 03 21:45:54.704614 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691924 6524 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 03 21:45:54.704614 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691926 6524 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 03 21:45:54.704614 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691929 6524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 03 21:45:54.704614 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691932 6524 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 03 21:45:54.704614 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691935 6524 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 03 21:45:54.704614 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691938 6524 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 03 21:45:54.704614 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691940 6524 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 03 21:45:54.704614 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691943 6524 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 03 21:45:54.704614 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691946 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 03 21:45:54.704614 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691949 6524 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 03 21:45:54.704614 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691951 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 03 21:45:54.704614 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691954 6524 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 03 21:45:54.704614 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691956 6524 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 03 21:45:54.705088 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691959 6524 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 03 21:45:54.705088 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.691961 6524 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 03 21:45:54.705088 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.691990 6524 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 03 21:45:54.705088 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692163 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 03 21:45:54.705088 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692170 6524 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 03 21:45:54.705088 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692173 6524 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 03 21:45:54.705088 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692176 6524 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 03 21:45:54.705088 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692179 6524 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 03 21:45:54.705088 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692183 6524 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 03 21:45:54.705088 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692186 6524 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 03 21:45:54.705088 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692188 6524 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 03 21:45:54.705088 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692191 6524 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 03 21:45:54.705088 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692194 6524 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 03 21:45:54.705088 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692196 6524 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 03 21:45:54.705088 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692199 6524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 03 21:45:54.705088 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692201 6524 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 03 21:45:54.705481 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692204 6524 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 03 21:45:54.705481 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692207 6524 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 03 21:45:54.705481 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692210 6524 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 03 21:45:54.705481 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692212 6524 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 03 21:45:54.705481 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692215 6524 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 03 21:45:54.705481 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692218 6524 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 03 21:45:54.705481 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692221 6524 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 03 21:45:54.705481 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692224 6524 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 03 21:45:54.705481 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692226 6524 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 03 21:45:54.705481 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692229 6524 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 03 21:45:54.705481 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692232 6524 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 03 21:45:54.705481 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692235 6524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 03 21:45:54.705481 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692238 6524 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 03 21:45:54.705481 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692240 6524 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 03 21:45:54.705481 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692243 6524 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 03 21:45:54.705481 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692245 6524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 03 21:45:54.705481 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692248 6524 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 03 21:45:54.705481 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692250 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 03 21:45:54.705481 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692253 6524 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 03 21:45:54.705481 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692256 6524 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 03 21:45:54.705965 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692259 6524 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 03 21:45:54.705965 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692269 6524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 03 21:45:54.705965 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692272 6524 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 03 21:45:54.705965 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692275 6524 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 03 21:45:54.705965 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692277 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 03 21:45:54.705965 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692281 6524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 03 21:45:54.705965 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692284 6524 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 03 21:45:54.705965 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692287 6524 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 03 21:45:54.705965 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692289 6524 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 03 21:45:54.705965 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692292 6524 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 03 21:45:54.705965 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692294 6524 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 03 21:45:54.705965 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692297 6524 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 03 21:45:54.705965 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692299 6524 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 03 21:45:54.705965 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692302 6524 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 03 21:45:54.705965 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692304 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 03 21:45:54.705965 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692307 6524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 03 21:45:54.705965 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692310 6524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 03 21:45:54.705965 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692312 6524 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 03 21:45:54.705965 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692315 6524 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 03 21:45:54.705965 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692318 6524 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 03 21:45:54.706469 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692321 6524 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 03 21:45:54.706469 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692323 6524 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 03 21:45:54.706469 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692326 6524 feature_gate.go:386] unrecognized feature gate: EVPN Nov 03 21:45:54.706469 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692328 6524 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 03 21:45:54.706469 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692331 6524 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 03 21:45:54.706469 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692334 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 03 21:45:54.706469 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692336 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 03 21:45:54.706469 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692339 6524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 03 21:45:54.706469 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692342 6524 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 03 21:45:54.706469 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692345 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 03 21:45:54.706469 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692347 6524 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 03 21:45:54.706469 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692350 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 03 21:45:54.706469 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692353 6524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 03 21:45:54.706469 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692355 6524 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 03 21:45:54.706469 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692358 6524 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 03 21:45:54.706469 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692360 6524 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 03 21:45:54.706469 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692363 6524 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 03 21:45:54.706469 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692365 6524 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 03 21:45:54.706469 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692369 6524 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 03 21:45:54.706948 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692371 6524 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 03 21:45:54.706948 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692374 6524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 03 21:45:54.706948 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692376 6524 feature_gate.go:386] unrecognized feature gate: Example Nov 03 21:45:54.706948 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692379 6524 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 03 21:45:54.706948 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692381 6524 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 03 21:45:54.706948 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692384 6524 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 03 21:45:54.706948 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692387 6524 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 03 21:45:54.706948 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692389 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 03 21:45:54.706948 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692392 6524 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 03 21:45:54.706948 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692394 6524 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 03 21:45:54.706948 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692397 6524 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 03 21:45:54.706948 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692401 6524 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 03 21:45:54.706948 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692405 6524 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 03 21:45:54.706948 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692408 6524 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 03 21:45:54.706948 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692412 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 03 21:45:54.706948 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692414 6524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 03 21:45:54.706948 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692417 6524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 03 21:45:54.706948 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692419 6524 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 03 21:45:54.706948 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692422 6524 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 03 21:45:54.707612 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692425 6524 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 03 21:45:54.707612 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692427 6524 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 03 21:45:54.707612 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692430 6524 feature_gate.go:386] unrecognized feature gate: Example2 Nov 03 21:45:54.707612 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692432 6524 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 03 21:45:54.707612 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692435 6524 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 03 21:45:54.707612 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692437 6524 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 03 21:45:54.707612 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692440 6524 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 03 21:45:54.707612 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692442 6524 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 03 21:45:54.707612 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692445 6524 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 03 21:45:54.707612 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692447 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 03 21:45:54.707612 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692450 6524 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 03 21:45:54.707612 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692452 6524 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 03 21:45:54.707612 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692455 6524 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 03 21:45:54.707612 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692458 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 03 21:45:54.707612 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692460 6524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 03 21:45:54.707612 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.692489 6524 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 03 21:45:54.708136 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692524 6524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 03 21:45:54.708136 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692529 6524 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 03 21:45:54.708136 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692532 6524 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 03 21:45:54.708136 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692534 6524 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 03 21:45:54.708136 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692537 6524 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 03 21:45:54.708136 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692540 6524 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 03 21:45:54.708136 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692543 6524 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 03 21:45:54.708136 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692546 6524 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 03 21:45:54.708136 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692548 6524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 03 21:45:54.708136 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692551 6524 feature_gate.go:386] unrecognized feature gate: Example Nov 03 21:45:54.708136 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692554 6524 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 03 21:45:54.708136 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692557 6524 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 03 21:45:54.708136 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692559 6524 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 03 21:45:54.708136 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692562 6524 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 03 21:45:54.708136 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692565 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 03 21:45:54.708136 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692567 6524 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 03 21:45:54.708136 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692570 6524 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 03 21:45:54.708136 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692573 6524 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 03 21:45:54.708770 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692577 6524 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 03 21:45:54.708770 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692581 6524 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 03 21:45:54.708770 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692584 6524 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 03 21:45:54.708770 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692587 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 03 21:45:54.708770 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692590 6524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 03 21:45:54.708770 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692593 6524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 03 21:45:54.708770 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692595 6524 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 03 21:45:54.708770 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692598 6524 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 03 21:45:54.708770 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692601 6524 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 03 21:45:54.708770 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692604 6524 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 03 21:45:54.708770 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692606 6524 feature_gate.go:386] unrecognized feature gate: Example2 Nov 03 21:45:54.708770 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692609 6524 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 03 21:45:54.708770 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692612 6524 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 03 21:45:54.708770 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692615 6524 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 03 21:45:54.708770 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692619 6524 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 03 21:45:54.708770 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692622 6524 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 03 21:45:54.708770 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692625 6524 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 03 21:45:54.708770 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692627 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 03 21:45:54.708770 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692630 6524 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 03 21:45:54.709405 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692632 6524 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 03 21:45:54.709405 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692635 6524 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 03 21:45:54.709405 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692638 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 03 21:45:54.709405 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692640 6524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 03 21:45:54.709405 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692643 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 03 21:45:54.709405 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692646 6524 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 03 21:45:54.709405 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692649 6524 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 03 21:45:54.709405 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692652 6524 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 03 21:45:54.709405 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692654 6524 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 03 21:45:54.709405 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692657 6524 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 03 21:45:54.709405 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692660 6524 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 03 21:45:54.709405 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692662 6524 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 03 21:45:54.709405 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692665 6524 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 03 21:45:54.709405 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692667 6524 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 03 21:45:54.709405 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692670 6524 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 03 21:45:54.709405 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692673 6524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 03 21:45:54.709405 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692676 6524 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 03 21:45:54.709405 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692678 6524 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 03 21:45:54.709405 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692680 6524 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 03 21:45:54.709405 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692683 6524 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 03 21:45:54.710320 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692686 6524 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 03 21:45:54.710320 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692688 6524 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 03 21:45:54.710320 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692691 6524 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 03 21:45:54.710320 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692693 6524 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 03 21:45:54.710320 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692696 6524 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 03 21:45:54.710320 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692699 6524 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 03 21:45:54.710320 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692702 6524 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 03 21:45:54.710320 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692706 6524 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 03 21:45:54.710320 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692709 6524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 03 21:45:54.710320 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692711 6524 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 03 21:45:54.710320 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692714 6524 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 03 21:45:54.710320 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692716 6524 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 03 21:45:54.710320 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692719 6524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 03 21:45:54.710320 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692721 6524 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 03 21:45:54.710320 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692724 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 03 21:45:54.710320 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692727 6524 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 03 21:45:54.710320 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692729 6524 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 03 21:45:54.710320 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692733 6524 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 03 21:45:54.710320 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692737 6524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 03 21:45:54.711061 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692740 6524 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 03 21:45:54.711061 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692743 6524 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 03 21:45:54.711061 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692746 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 03 21:45:54.711061 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692748 6524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 03 21:45:54.711061 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692751 6524 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 03 21:45:54.711061 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692753 6524 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 03 21:45:54.711061 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692756 6524 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 03 21:45:54.711061 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692758 6524 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 03 21:45:54.711061 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692761 6524 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 03 21:45:54.711061 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692763 6524 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 03 21:45:54.711061 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692766 6524 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 03 21:45:54.711061 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692768 6524 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 03 21:45:54.711061 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692771 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 03 21:45:54.711061 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692774 6524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 03 21:45:54.711061 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692776 6524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 03 21:45:54.711061 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692778 6524 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 03 21:45:54.711061 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692781 6524 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 03 21:45:54.711061 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692783 6524 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 03 21:45:54.711061 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692786 6524 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 03 21:45:54.711061 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692789 6524 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 03 21:45:54.711606 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692794 6524 feature_gate.go:386] unrecognized feature gate: EVPN Nov 03 21:45:54.711606 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692796 6524 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 03 21:45:54.711606 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692799 6524 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 03 21:45:54.711606 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692802 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 03 21:45:54.711606 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692804 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 03 21:45:54.711606 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692807 6524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 03 21:45:54.711606 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692809 6524 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 03 21:45:54.711606 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692812 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 03 21:45:54.711606 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692815 6524 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 03 21:45:54.711606 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:45:54.692819 6524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 03 21:45:54.711606 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.692874 6524 server.go:957] "Client rotation is on, will bootstrap in background" Nov 03 21:45:54.711606 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.693905 6524 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary" Nov 03 21:45:54.711606 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.693967 6524 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem" Nov 03 21:45:54.711606 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.694172 6524 server.go:1014] "Starting client certificate rotation" Nov 03 21:45:54.711606 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.694312 6524 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Nov 03 21:45:54.711606 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.694364 6524 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-02 21:39:44 +0000 UTC" deadline="2028-05-29 20:16:31.890949741 +0000 UTC" Nov 03 21:45:54.712007 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.694413 6524 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13750h30m37.196539914s" Nov 03 21:45:54.712007 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.694726 6524 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 03 21:45:54.712007 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.695843 6524 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 03 21:45:54.712007 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.697300 6524 log.go:25] "Validated CRI v1 runtime API" Nov 03 21:45:54.712007 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.699681 6524 log.go:25] "Validated CRI v1 image API" Nov 03 21:45:54.712007 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.700386 6524 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Nov 03 21:45:54.712007 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.702723 6524 fs.go:135] Filesystem UUIDs: map[7141210b-eb02-42fb-9c77-7dd78df61112:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 8e1fc453-7c05-41e1-85fc-af7bbf0c43f3:/dev/nvme0n1p3] Nov 03 21:45:54.712280 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.702743 6524 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/06948bda24b584813e431ecca9ab2c381f1466d883d2c21d599a46a87854cb4d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/06948bda24b584813e431ecca9ab2c381f1466d883d2c21d599a46a87854cb4d/userdata/shm major:0 minor:207 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0b349470d1bf258e06b2810cc311aff71f9653d22e5f6598ec3f038f6646a0bb/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0b349470d1bf258e06b2810cc311aff71f9653d22e5f6598ec3f038f6646a0bb/userdata/shm major:0 minor:249 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/106c43f9fcf1a1c3acc5d75e8010b45678a9a16af360e8ab411f4c897fd8dbc2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/106c43f9fcf1a1c3acc5d75e8010b45678a9a16af360e8ab411f4c897fd8dbc2/userdata/shm major:0 minor:274 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/12d4118e93b819d9eb3d110e90fce86db0b42341bf9196e7600928553ea1077a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/12d4118e93b819d9eb3d110e90fce86db0b42341bf9196e7600928553ea1077a/userdata/shm major:0 minor:385 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/14b5af6921fb45781a99bda05244046395650b5904b97226a33bfda38089f77f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/14b5af6921fb45781a99bda05244046395650b5904b97226a33bfda38089f77f/userdata/shm major:0 minor:77 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1b370a6c830e98f01be8c325b5aecac0f16f4ab08328435306f38e3b5b58aea7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1b370a6c830e98f01be8c325b5aecac0f16f4ab08328435306f38e3b5b58aea7/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3bb48199a050488f952dbd7eefe0990c61e8638c924e084fbe0e9105710f7be6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3bb48199a050488f952dbd7eefe0990c61e8638c924e084fbe0e9105710f7be6/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3cb26baa6890fd84328a69c267cd625706f3fd3c705d645908c5d46f151828b3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3cb26baa6890fd84328a69c267cd625706f3fd3c705d645908c5d46f151828b3/userdata/shm major:0 minor:323 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3eb641ca1e372ef33112a45ca00729446a14092b592b2c08fe59d58f33361bf4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3eb641ca1e372ef33112a45ca00729446a14092b592b2c08fe59d58f33361bf4/userdata/shm major:0 minor:285 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3f1b9e13ddbf6ca5a0b73da16e9f34643e98cbe21681f3aa9c6c28106302b181/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3f1b9e13ddbf6ca5a0b73da16e9f34643e98cbe21681f3aa9c6c28106302b181/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/43c10bc2bd2b94e42dbe0ea4ce382277116893404dc90d6a70e48102ec9fcbeb/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/43c10bc2bd2b94e42dbe0ea4ce382277116893404dc90d6a70e48102ec9fcbeb/userdata/shm major:0 minor:431 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/484d94934d6014e3bd2e5fcfb68db6488c20d2d1ec8cefbc9075eeab26519f20/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/484d94934d6014e3bd2e5fcfb68db6488c20d2d1ec8cefbc9075eeab26519f20/userdata/shm major:0 minor:84 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5909793b2fc1d44e499b7ee12a8b1b1ff8d06a17a4790707c3eb3ab97c2259f0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5909793b2fc1d44e499b7ee12a8b1b1ff8d06a17a4790707c3eb3ab97c2259f0/userdata/shm major:0 minor:209 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8783df8686233a4ddb7fe45426c063a89d5e82fbd6273021e42a43027efbd5de/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8783df8686233a4ddb7fe45426c063a89d5e82fbd6273021e42a43027efbd5de/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/93cbd093a41d6f8280f48bbb0d4906b6cf9bcedd415031f1e155400be7c35b12/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/93cbd093a41d6f8280f48bbb0d4906b6cf9bcedd415031f1e155400be7c35b12/userdata/shm major:0 minor:61 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9d446131f627080dcfe0c7098bf0cc9b0e356964bd3cca77e96faf8eba808bb3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9d446131f627080dcfe0c7098bf0cc9b0e356964bd3cca77e96faf8eba808bb3/userdata/shm major:0 minor:204 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ab4491eda7139fd31fdc7848d7f296f2c1f81b905d51c048f12a2ae3ea71da00/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ab4491eda7139fd31fdc7848d7f296f2c1f81b905d51c048f12a2ae3ea71da00/userdata/shm major:0 minor:429 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b251d8a015579cceb541952f1ebcb8e675394a3ad80016d966f745b49bbfb30c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b251d8a015579cceb541952f1ebcb8e675394a3ad80016d966f745b49bbfb30c/userdata/shm major:0 minor:433 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c244c1362b3ac28277d760dd45ee267608cffbbcd8811e1b3de0d2f5e23796df/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c244c1362b3ac28277d760dd45ee267608cffbbcd8811e1b3de0d2f5e23796df/userdata/shm major:0 minor:60 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/cae35dc92e36aaa91e303dd7f694cff5e2cc1a2b8830c0eda5a4496a101f574e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/cae35dc92e36aaa91e303dd7f694cff5e2cc1a2b8830c0eda5a4496a101f574e/userdata/shm major:0 minor:62 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/da3b1b4886307f48cd0e54b04214c86fafa13ec3b4d3891c843639464ccd9707/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/da3b1b4886307f48cd0e54b04214c86fafa13ec3b4d3891c843639464ccd9707/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ecfdcbf875981eb0a48d60cbc1003e95e865da3fd3fd6dec53c35397b6308e43/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ecfdcbf875981eb0a48d60cbc1003e95e865da3fd3fd6dec53c35397b6308e43/userdata/shm major:0 minor:260 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/00c924c1-ac0d-49c8-8610-3b837fbb8277/volumes/kubernetes.io~projected/kube-api-access-phmrg:{mountpoint:/var/lib/kubelet/pods/00c924c1-ac0d-49c8-8610-3b837fbb8277/volumes/kubernetes.io~projected/kube-api-access-phmrg major:0 minor:284 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/00c924c1-ac0d-49c8-8610-3b837fbb8277/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/00c924c1-ac0d-49c8-8610-3b837fbb8277/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:280 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/00c924c1-ac0d-49c8-8610-3b837fbb8277/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/00c924c1-ac0d-49c8-8610-3b837fbb8277/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy major:0 minor:281 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/00c924c1-ac0d-49c8-8610-3b837fbb8277/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics:{mountpoint:/var/lib/kubelet/pods/00c924c1-ac0d-49c8-8610-3b837fbb8277/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics major:0 minor:283 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/00c924c1-ac0d-49c8-8610-3b837fbb8277/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules:{mountpoint:/var/lib/kubelet/pods/00c924c1-ac0d-49c8-8610-3b837fbb8277/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules major:0 minor:278 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/00c924c1-ac0d-49c8-8610-3b837fbb8277/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/00c924c1-ac0d-49c8-8610-3b837fbb8277/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web major:0 minor:279 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/00c924c1-ac0d-49c8-8610-3b837fbb8277/volumes/kubernetes.io~secret/secret-thanos-querier-tls:{mountpoint:/var/lib/kubelet/pods/00c924c1-ac0d-49c8-8610-3b837fbb8277/volumes/kubernetes.io~secret/secret-thanos-querier-tls major:0 minor:282 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0db6d946-a1d8-4459-a883-6f93f7e1182a/volumes/kubernetes.io~projected/kube-api-access-6c6w8:{mountpoint:/var/lib/kubelet/pods/0db6d946-a1d8-4459-a883-6f93f7e1182a/volumes/kubernetes.io~projected/kube-api-access-6c6w8 major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volume-subpaths/web-config/config-reloader/2:{mountpoint:/var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volume-subpaths/web-config/config-reloader/2 major:0 minor:383 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volume-subpaths/web-config/init-config-reloader/2:{mountpoint:/var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volume-subpaths/web-config/init-config-reloader/2 major:0 minor:383 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volume-subpaths/web-config/prometheus/15:{mountpoint:/var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volume-subpaths/web-config/prometheus/15 major:0 minor:383 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volumes/kubernetes.io~empty-dir/config-out major:0 minor:375 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volumes/kubernetes.io~projected/kube-api-access-mlgsq:{mountpoint:/var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volumes/kubernetes.io~projected/kube-api-access-mlgsq major:0 minor:384 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volumes/kubernetes.io~projected/tls-assets major:0 minor:379 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volumes/kubernetes.io~secret/config:{mountpoint:/var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volumes/kubernetes.io~secret/config major:0 minor:377 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:381 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volumes/kubernetes.io~secret/secret-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volumes/kubernetes.io~secret/secret-kube-rbac-proxy major:0 minor:382 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volumes/kubernetes.io~secret/secret-metrics-client-certs:{mountpoint:/var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volumes/kubernetes.io~secret/secret-metrics-client-certs major:0 minor:376 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web major:0 minor:378 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls:{mountpoint:/var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls major:0 minor:374 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls:{mountpoint:/var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls major:0 minor:380 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file:{mountpoint:/var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file major:0 minor:370 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volumes/kubernetes.io~secret/web-config major:0 minor:383 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/219625cd-f1a4-4e51-90ee-58f0fc5939bb/volumes/kubernetes.io~projected/kube-api-access-kl5nr:{mountpoint:/var/lib/kubelet/pods/219625cd-f1a4-4e51-90ee-58f0fc5939bb/volumes/kubernetes.io~projected/kube-api-access-kl5nr major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/32fd1d86-45ed-4a7d-894e-3990980bc237/volumes/kubernetes.io~projected/kube-api-access-m8h9g:{mountpoint:/var/lib/kubelet/pods/32fd1d86-45ed-4a7d-894e-3990980bc237/volumes/kubernetes.io~projected/kube-api-access-m8h9g major:0 minor:205 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/32fd1d86-45ed-4a7d-894e-3990980bc237/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/32fd1d86-45ed-4a7d-894e-3990980bc237/volumes/kubernetes.io~secret/cert major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/48d8b783-b602-47ef-9b9b-f2416d448908/volumes/kubernetes.io~projected/kube-api-access-k8fqs:{mountpoint:/var/lib/kubelet/pods/48d8b783-b602-47ef-9b9b-f2416d448908/volumes/kubernetes.io~projected/kube-api-access-k8fqs major:0 minor:54 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5921d9cf-54a7-43bd-b697-c5ccf9f39135/volumes/kubernetes.io~projected/kube-api-access-t7m8x:{mountpoint:/var/lib/kubelet/pods/5921d9cf-54a7-43bd-b697-c5ccf9f39135/volumes/kubernetes.io~projected/kube-api-access-t7m8x major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5921d9cf-54a7-43bd-b697-c5ccf9f39135/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/5921d9cf-54a7-43bd-b697-c5ccf9f39135/volumes/kubernetes.io~secret/metrics-tls major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5d7e7316-1bd4-42cf-8519-544c1e2a4526/volumes/kubernetes.io~projected/kube-api-access-zh4kw:{mountpoint:/var/lib/kubelet/pods/5d7e7316-1bd4-42cf-8519-544c1e2a4526/volumes/kubernetes.io~projected/kube-api-access-zh4kw major:0 minor:428 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c/volumes/kubernetes.io~projected/kube-api-access-csdhx:{mountpoint:/var/lib/kubelet/pods/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c/volumes/kubernetes.io~projected/kube-api-access-csdhx major:0 minor:48 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/76122905-15b4-4f4e-97b9-9f73a7675e36/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/76122905-15b4-4f4e-97b9-9f73a7675e36/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/76122905-15b4-4f4e-97b9-9f73a7675e36/volumes/kubernetes.io~projected/kube-api-access-tmr4b:{mountpoint:/var/lib/kubelet/pods/76122905-15b4-4f4e-97b9-9f73a7675e36/volumes/kubernetes.io~projected/kube-api-access-tmr4b major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/76122905-15b4-4f4e-97b9-9f73a7675e36/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/76122905-15b4-4f4e-97b9-9f73a7675e36/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:49 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/811514f5-baff-4fc1-9e92-28f27de7b0b7/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/811514f5-baff-4fc1-9e92-28f27de7b0b7/volumes/kubernetes.io~secret/agent-certs major:0 minor:52 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/866ddb5a-c929-40c1-911d-ca40c5405723/volumes/kubernetes.io~projected/kube-api-access-n5kf4:{mountpoint:/var/lib/kubelet/pods/866ddb5a-c929-40c1-911d-ca40c5405723/volumes/kubernetes.io~projected/kube-api-access-n5kf4 major:0 minor:259 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/866ddb5a-c929-40c1-911d-ca40c5405723/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/866ddb5a-c929-40c1-911d-ca40c5405723/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:258 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/866ddb5a-c929-40c1-911d-ca40c5405723/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/866ddb5a-c929-40c1-911d-ca40c5405723/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:257 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/94aaa3ac-c7fc-4f87-878e-8ef79127488b/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/94aaa3ac-c7fc-4f87-878e-8ef79127488b/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:423 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9c970a61-3ebf-48c9-a05b-688accd0b18a/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/9c970a61-3ebf-48c9-a05b-688accd0b18a/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:50 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9c970a61-3ebf-48c9-a05b-688accd0b18a/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/9c970a61-3ebf-48c9-a05b-688accd0b18a/volumes/kubernetes.io~empty-dir/tmp major:0 minor:51 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9c970a61-3ebf-48c9-a05b-688accd0b18a/volumes/kubernetes.io~projected/kube-api-access-9k8xx:{mountpoint:/var/lib/kubelet/pods/9c970a61-3ebf-48c9-a05b-688accd0b18a/volumes/kubernetes.io~projected/kube-api-access-9k8xx major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9f5c183e-d58a-4621-940c-04a7ae0bafc9/volumes/kubernetes.io~projected/kube-api-access-rg4r2:{mountpoint:/var/lib/kubelet/pods/9f5c183e-d58a-4621-940c-04a7ae0bafc9/volumes/kubernetes.io~projected/kube-api-access-rg4r2 major:0 minor:47 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a2ccde5e-94a2-47ac-aff9-65248e4a6c92/volume-subpaths/cluster-tls-config/alertmanager/10:{mountpoint:/var/lib/kubelet/pods/a2ccde5e-94a2-47ac-aff9-65248e4a6c92/volume-subpaths/cluster-tls-config/alertmanager/10 major:0 minor:268 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a2ccde5e-94a2-47ac-aff9-65248e4a6c92/volume-subpaths/web-config/alertmanager/9:{mountpoint:/var/lib/kubelet/pods/a2ccde5e-94a2-47ac-aff9-65248e4a6c92/volume-subpaths/web-config/alertmanager/9 major:0 minor:262 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a2ccde5e-94a2-47ac-aff9-65248e4a6c92/volume-subpaths/web-config/config-reloader/6:{mountpoint:/var/lib/kubelet/pods/a2ccde5e-94a2-47ac-aff9-65248e4a6c92/volume-subpaths/web-config/config-reloader/6 major:0 minor:262 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a2ccde5e-94a2-47ac-aff9-65248e4a6c92/volume-subpaths/web-config/init-config-reloader/6:{mountpoint:/var/lib/kubelet/pods/a2ccde5e-94a2-47ac-aff9-65248e4a6c92/volume-subpaths/web-config/init-config-reloader/6 major:0 minor:262 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a2ccde5e-94a2-47ac-aff9-65248e4a6c92/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/a2ccde5e-94a2-47ac-aff9-65248e4a6c92/volumes/kubernetes.io~empty-dir/config-out major:0 minor:266 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a2ccde5e-94a2-47ac-aff9-65248e4a6c92/volumes/kubernetes.io~projected/kube-api-access-s7g58:{mountpoint:/var/lib/kubelet/pods/a2ccde5e-94a2-47ac-aff9-65248e4a6c92/volumes/kubernetes.io~projected/kube-api-access-s7g58 major:0 minor:271 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a2ccde5e-94a2-47ac-aff9-65248e4a6c92/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/a2ccde5e-94a2-47ac-aff9-65248e4a6c92/volumes/kubernetes.io~projected/tls-assets major:0 minor:269 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a2ccde5e-94a2-47ac-aff9-65248e4a6c92/volumes/kubernetes.io~secret/cluster-tls-config:{mountpoint:/var/lib/kubelet/pods/a2ccde5e-94a2-47ac-aff9-65248e4a6c92/volumes/kubernetes.io~secret/cluster-tls-config major:0 minor:268 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a2ccde5e-94a2-47ac-aff9-65248e4a6c92/volumes/kubernetes.io~secret/config-volume:{mountpoint:/var/lib/kubelet/pods/a2ccde5e-94a2-47ac-aff9-65248e4a6c92/volumes/kubernetes.io~secret/config-volume major:0 minor:265 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a2ccde5e-94a2-47ac-aff9-65248e4a6c92/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/a2ccde5e-94a2-47ac-aff9-65248e4a6c92/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy major:0 minor:263 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a2ccde5e-94a2-47ac-aff9-65248e4a6c92/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric:{mountpoint:/var/lib/kubelet/pods/a2ccde5e-94a2-47ac-aff9-65248e4a6c92/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric major:0 minor:264 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a2ccde5e-94a2-47ac-aff9-65248e4a6c92/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/a2ccde5e-94a2-47ac-aff9-65248e4a6c92/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web major:0 minor:267 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a2ccde5e-94a2-47ac-aff9-65248e4a6c92/volumes/kubernetes.io~secret/secret-alertmanager-main-tls:{mountpoint:/var/lib/kubelet/pods/a2ccde5e-94a2-47ac-aff9-65248e4a6c92/volumes/kubernetes.io~secret/secret-alertmanager-main-tls major:0 minor:270 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a2ccde5e-94a2-47ac-aff9-65248e4a6c92/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/a2ccde5e-94a2-47ac-aff9-65248e4a6c92/volumes/kubernetes.io~secret/web-config major:0 minor:262 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a440580e-b0bd-49e2-a33d-576118e049ae/volumes/kubernetes.io~projected/kube-api-access-lqplq:{mountpoint:/var/lib/kubelet/pods/a440580e-b0bd-49e2-a33d-576118e049ae/volumes/kubernetes.io~projected/kube-api-access-lqplq major:0 minor:53 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a6cf0029-66be-4d86-ae8e-713d173a113e/volumes/kubernetes.io~projected/kube-api-access-zfkkl:{mountpoint:/var/lib/kubelet/pods/a6cf0029-66be-4d86-ae8e-713d173a113e/volumes/kubernetes.io~projected/kube-api-access-zfkkl major:0 minor:322 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a6cf0029-66be-4d86-ae8e-713d173a113e/volumes/kubernetes.io~secret/monitoring-plugin-cert:{mountpoint:/var/lib/kubelet/pods/a6cf0029-66be-4d86-ae8e-713d173a113e/volumes/kubernetes.io~secret/monitoring-plugin-cert major:0 minor:318 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bfd8df86-d3d0-4576-afd0-fa742a6a3aad/volumes/kubernetes.io~projected/kube-api-access-jl4cx:{mountpoint:/var/lib/kubelet/pods/bfd8df86-d3d0-4576-afd0-fa742a6a3aad/volumes/kubernetes.io~projected/kube-api-access-jl4cx major:0 minor:239 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bfd8df86-d3d0-4576-afd0-fa742a6a3aad/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/bfd8df86-d3d0-4576-afd0-fa742a6a3aad/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:238 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bfd8df86-d3d0-4576-afd0-fa742a6a3aad/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/bfd8df86-d3d0-4576-afd0-fa742a6a3aad/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:246 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e652c3ec-d5ac-43f9-ade5-d3da32041cee/volumes/kubernetes.io~projected/kube-api-access-p7xhd:{mountpoint:/var/lib/kubelet/pods/e652c3ec-d5ac-43f9-ade5-d3da32041cee/volumes/kubernetes.io~projected/kube-api-access-p7xhd major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e652c3ec-d5ac-43f9-ade5-d3da32041cee/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/e652c3ec-d5ac-43f9-ade5-d3da32041cee/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/eb5f720e-f465-439f-ba32-4f595561f9f6/volumes/kubernetes.io~projected/kube-api-access-pgzcx:{mountpoint:/var/lib/kubelet/pods/eb5f720e-f465-439f-ba32-4f595561f9f6/volumes/kubernetes.io~projected/kube-api-access-pgzcx major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/eb5f720e-f465-439f-ba32-4f595561f9f6/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/eb5f720e-f465-439f-ba32-4f595561f9f6/volumes/kubernetes.io~secret/metrics-certs major:0 minor:427 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/4e56f9f268862e00e26e83271c7c53d113d06a40902755317eb5d02c2e5a5fa2/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/59120c8efdf3359ade1167744868117c96f17e39724795c3d3c4859b96e4489b/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/2a781e4e59c5759f112d6989829a02dc0d7af8a15725febcb323620a21fdd304/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-109:{mountpoint:/var/lib/containers/storage/overlay/e6bac6c5e3dabc4465f5491e0fd2420d5a8d149bba35f7764827864796cb2ea8/merged major:0 minor:109 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/19e0b8a605d26bf902d1151f87e4d6eaa160d0d0ad3a22c45b9ba0106c2a1ae5/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/0ecbcad05fe2e3c6ac0da5c359b6d9cb1a726cb3939dd91cdd6722b13fbe6761/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/5535539e3e2f0ef3a03f9fbeb81247fbdc66cf449834a3523440e8fe1ee08811/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/8ac909c59c4335ae2edc060beb2490489ec428067d0ce829a28c5e8bffbc107a/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/7ee880b0612dd5e8681614e5b1653bd3c07c09fc85d2c5354f691e421111ade5/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/ae94068ef574f51acff51a113742576ea6c0f52cadcd77ca0b28f0aec8b191f2/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-139:{mountpoint:/var/lib/containers/storage/overlay/dd0416a600c17563b5be3d5be573af4053e6ca15e2964abb0691bbdc60a2be48/merged major:0 minor:139 fsType:overlay blockSize:0} overlay_0-146:{mountpoint:/var/lib/containers/storage/overlay/8d659450565e58405e756c78d6bf7cce2f2dbe9c0503c5d94182605d472d0bd1/merged major:0 minor:146 fsType:overlay blockSize:0} overlay_0-149:{mountpoint:/var/lib/containers/storage/overlay/abe8e7e073f1927af6761514bd130b57bc9c233f7fe994e6e9001eae4ef56505/merged major:0 minor:149 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/244966cb85caa287b266ad0f49b3e35bb460086d09578beb0d8a731c02b4a84e/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-159:{mountpoint:/var/lib/containers/storage/overlay/f96fd8f9655508a772101bc0a0312a4a83aac6d9229fb57f96f3b63812ac424f/merged major:0 minor:159 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/50888a1874478d9b3ff9fc60db7b36930769bfeb2364809f463ef61169ca70b0/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/1a79a6670e93d729c3aa5c45e3cf995f4735c78865c7e2436233dc8352d1d03b/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/dce5a0023eea8e34dc937beff55c3e59eab7be226c78099635e85e2afe89a8ce/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/ba33fd6e40251f3135a937d9fc294d4cc1e3b744a26444dda24933eda5ec4081/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/a44fa1588c0621876d86c465b8c3d0f56c8ccafc68e3057bd1f0a2506409fff4/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/5ea0ccaef262644203e1c16b322bae52d874b7f8ee6b1fb447a46c909838380e/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/996a0ad6c207248c895ca36c11650238f7596919d9707331fb5fbfdafdebfe7d/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-213:{mountpoint:/var/lib/containers/storage/overlay/caa7913ddd531445417c300653aecff32f79a04f5e05e58d719fd71e7520bd22/merged major:0 minor:213 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/abc683aa303515c8e994a2ad02fc9ac49630d222e7681514bd9a728251c64958/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-217:{mountpoint:/var/lib/containers/storage/overlay/02d6014ac992de499b12605e33ff9a715755cb88381cd6e50efcf53f2f1fc862/merged major:0 minor:217 fsType:overlay blockSize:0} overlay_0-222:{mountpoint:/var/lib/containers/storage/overlay/256ef8a64cd963a55f3884d46311d91993f8f45ccd4fe55ffc179f849aeaf261/merged major:0 minor:222 fsType:overlay blockSize:0} overlay_0-225:{mountpoint:/var/lib/containers/storage/overlay/6592a7445122b7ab40ae8efef40226cc6bda02d3b6639f52c4785ea72ccaa67b/merged major:0 minor:225 fsType:overlay blockSize:0} overlay_0-230:{mountpoint:/var/lib/containers/storage/overlay/57d476c91d6521e06b5a73e874d82f753d50cc4b5bfc83f73ed2fb154cb1eade/merged major:0 minor:230 fsType:overlay blockSize:0} overlay_0-236:{mountpoint:/var/lib/containers/storage/overlay/adc51c44ae91c1c92588a25a3328af9a13cc986d10d2c82f3ad3726ce3a7de34/merged major:0 minor:236 fsType:overlay blockSize:0} overlay_0-247:{mountpoint:/var/lib/containers/storage/overlay/26d5e9186e082d5ab3018ad4471f216f03a2ee13421b6c4df13e616f7c638710/merged major:0 minor:247 fsType:overlay blockSize:0} overlay_0-255:{mountpoint:/var/lib/containers/storage/overlay/50d7c56b7998e0e62da3649ff9eac6e93500fc07f459b88e44a9dcb660edd402/merged major:0 minor:255 fsType:overlay blockSize:0} overlay_0-272:{mountpoint:/var/lib/containers/storage/overlay/571b2ef8ddd03e0b2e043cb8e3d46fb5b68fdf8b9a4c5312449a8e0ffb5e1ff1/merged major:0 minor:272 fsType:overlay blockSize:0} overlay_0-276:{mountpoint:/var/lib/containers/storage/overlay/6f0774926269ac0e40dbb629710309c9fbc0d3dccaa910201bbb2f9e6f0e4d16/merged major:0 minor:276 fsType:overlay blockSize:0} overlay_0-287:{mountpoint:/var/lib/containers/storage/overlay/1f178405ac4fd09149193c60b6a471ac973972cdddf641bdded7b8be0651c6ea/merged major:0 minor:287 fsType:overlay blockSize:0} overlay_0-289:{mountpoint:/var/lib/containers/storage/overlay/dc6c8235541aadafb29eedad563031473d6c6d798d9fc661cf3454360b5823e6/merged major:0 minor:289 fsType:overlay blockSize:0} overlay_0-294:{mountpoint:/var/lib/containers/storage/overlay/7d4bdfa25614ec1426631e29346a8f43ec7e95e693210d96ec5340550a81ca35/merged major:0 minor:294 fsType:overlay blockSize:0} overlay_0-299:{mountpoint:/var/lib/containers/storage/overlay/4cd0f16c1030f71566c90f01b39b45ab1beeb1e16342de65412fbade5ee6eb61/merged major:0 minor:299 fsType:overlay blockSize:0} overlay_0-301:{mountpoint:/var/lib/containers/storage/overlay/072ad2d47e813f2ee3c0251a082e2b082e097e5efe46c018b3a0e308e0ccc609/merged major:0 minor:301 fsType:overlay blockSize:0} overlay_0-307:{mountpoint:/var/lib/containers/storage/overlay/0300d3c4cb779217a3427a933d7a58797559ae9e8fecc191978264438ee306e0/merged major:0 minor:307 fsType:overlay blockSize:0} overlay_0-313:{mountpoint:/var/lib/containers/storage/overlay/2cf1a5140023822239adb360f76f8cd8e3ccb0611894807de63c21922d83a3cb/merged major:0 minor:313 fsType:overlay blockSize:0} overlay_0-325:{mountpoint:/var/lib/containers/storage/overlay/5d91feaaf435a32496bdceca9fc589f98f4f100788af6949e2e834359d53afc8/merged major:0 minor:325 fsType:overlay blockSize:0} overlay_0-327:{mountpoint:/var/lib/containers/storage/overlay/9a2b1703256c9f5d8c5cbc0104f2df3c2a1dac9f23f69e7005aed59ba6986117/merged major:0 minor:327 fsType:overlay blockSize:0} overlay_0-329:{mountpoint:/var/lib/containers/storage/overlay/8c0325d6f8e846545b6facf05ee6434f9b8312725ef708ba40db542eb2cf4aa8/merged major:0 minor:329 fsType:overlay blockSize:0} overlay_0-334:{mountpoint:/var/lib/containers/storage/overlay/1492263b6cd4a17fce0e645917b8d985e6b77eb9b18a828ceb012b4d4c0f5ec5/merged major:0 minor:334 fsType:overlay blockSize:0} overlay_0-336:{mountpoint:/var/lib/containers/storage/overlay/bfb3c8b1d9953061d6a6b1e347c07c2133be2e6eeab007489045cfe7d69d3ba8/merged major:0 minor:336 fsType:overlay blockSize:0} overlay_0-345:{mountpoint:/var/lib/containers/storage/overlay/f8158bc1afc94b296dbc85ba9278a0d72ec8fb75004d03a014edcfda3eff663a/merged major:0 minor:345 fsType:overlay blockSize:0} overlay_0-347:{mountpoint:/var/lib/containers/storage/overlay/0c97e18455b00bc2477b063259eb5d508ff9f753ef98004af65125af5c2fb4e4/merged major:0 minor:347 fsType:overlay blockSize:0} overlay_0-355:{mountpoint:/var/lib/containers/storage/overlay/0e35ec73ace55707cad6f89357a2263b993a604dfa4eb70ddb0ed31f10e4f5fb/merged major:0 minor:355 fsType:overlay blockSize:0} overlay_0-360:{mountpoint:/var/lib/containers/storage/overlay/ed179efc75b96fcbf23cbfc2940aecc3a4118f26bf4fb688fca698998b1cb0e0/merged major:0 minor:360 fsType:overlay blockSize:0} overlay_0-365:{mountpoint:/var/lib/containers/storage/overlay/183e60bbbe1650c52d9e517327255498c56318ce2981c9657974f35b75050c75/merged major:0 minor:365 fsType:overlay blockSize:0} overlay_0-387:{mountpoint:/var/lib/containers/storage/overlay/2bd21e95b1fd99009ec4e14274baf0905a3079ccd63d11e51c08789d1f1bd405/merged major:0 minor:387 fsType:overlay blockSize:0} overlay_0-389:{mountpoint:/var/lib/containers/storage/overlay/fdfc806ec47304039c6cce32a9664781a233609390823e8f0ce66248b5e89eaf/merged major:0 minor:389 fsType:overlay blockSize:0} overlay_0-395:{mountpoint:/var/lib/containers/storage/overlay/0a0216fb6d5df34189f3d43612b88aa3274ad8e2ad42e52e2494b0962f90b974/merged major:0 minor:395 fsType:overlay blockSize:0} overlay_0-397:{mountpoint:/var/lib/containers/storage/overlay/1d578e4c3e8d52510149d5e10acb866155fbd920d80bc24de10dd2b2b71db577/merged major:0 minor:397 fsType:overlay blockSize:0} overlay_0-403:{mountpoint:/var/lib/containers/storage/overlay/48527dc65a351713e073d66ff377bc5c3d337ea5d82626aaf38936e7f1de0e96/merged major:0 minor:403 fsType:overlay blockSize:0} overlay_0-408:{mountpoint:/var/lib/containers/storage/overlay/4972419d9306155fa5ff8d426a628a7798c70d9e854d5635d250fbf7453318ab/merged major:0 minor:408 fsType:overlay blockSize:0} overlay_0-413:{mountpoint:/var/lib/containers/storage/overlay/582b504d502de5273993237d2a492fc75325fa6720b4dde8626a5d035ffef1ee/merged major:0 minor:413 fsType:overlay blockSize:0} overlay_0-418:{mountpoint:/var/lib/containers/storage/overlay/e9a92f6f65c3daaa0c76de6439d80675ed650611dbb718da7019969c04019325/merged major:0 minor:418 fsType:overlay blockSize:0} overlay_0-435:{mountpoint:/var/lib/containers/storage/overlay/f44fc1823dd52d1e9205422eae03ea240710f2f4e63468df89e3ead2b393edc5/merged major:0 minor:435 fsType:overlay blockSize:0} overlay_0-437:{mountpoint:/var/lib/containers/storage/overlay/5da04c73452dfc4a8b17c4cf24c22b3e8f2fa3e5a6963af012fd50217091193d/merged major:0 minor:437 fsType:overlay blockSize:0} overlay_0-439:{mountpoint:/var/lib/containers/storage/overlay/88257b4e2669f51fe823a708539b0942a831851b2cee35b8e7515e0d3701d447/merged major:0 minor:439 fsType:overlay blockSize:0} overlay_0-441:{mountpoint:/var/lib/containers/storage/overlay/5c7a6ca9a050aa1dcabe60061d642695d5a18ce473f01e6ae5549833c0a1b374/merged major:0 minor:441 fsType:overlay blockSize:0} overlay_0-443:{mountpoint:/var/lib/containers/storage/overlay/fc628239b0a63e24a063c5e0e7ba09cdf65e227ab1549ffaa5a201071eab9845/merged major:0 minor:443 fsType:overlay blockSize:0} overlay_0-449:{mountpoint:/var/lib/containers/storage/overlay/a929df8ea2beaeef1bb89204b2aad8a6dda4d6c6826957eb1494176b710bbe63/merged major:0 minor:449 fsType:overlay blockSize:0} overlay_0-454:{mountpoint:/var/lib/containers/storage/overlay/65b80cc9e5854c4bf888da48aa2b4ce3c83743b5659f394bcfaeda1134c83004/merged major:0 minor:454 fsType:overlay blockSize:0} overlay_0-66:{mountpoint:/var/lib/containers/storage/overlay/ba0f5f9e912ffe2e6bc9c8e8043d4a31cfff03420f43426c28af8a8877e75bcb/merged major:0 minor:66 fsType:overlay blockSize:0} overlay_0-68:{mountpoint:/var/lib/containers/storage/overlay/6e93bb290b5b801cef53ea469e20e10ca7d621d0f97f4aa65d418d0915dedd57/merged major:0 minor:68 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/03f8bc8c63425cb275bea66798e18d4757a03b2d0a79c9115fa42b8b02c11738/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/f6100a3c62f6b1769a47bb6d7df1a0fab10920c53f5348d306cbe3a867012be5/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/033e806caec8e2f0a9654286d5bb540d9324655d1d95c93d4595f92a22107b63/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/e25e5c16da388d5b4894acd33eba6d7e5ed954a0ca262ad34187ad8e08d6d040/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-86:{mountpoint:/var/lib/containers/storage/overlay/c0ff64f5ba9920c9a42b50a3d4071135c6b5b8f64f66bce4f5fda7f1b3c3bf73/merged major:0 minor:86 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/c2bdf2c59aaa222e017bc58109b09409567bc2a9f8a4d304a560d672da0f34c9/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/c76238e3c9744aeaf360e4ed92305d31228130044593cbca2e1e89e867b2bfdb/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/b75aa63a7c9ef936354fe294d9dddb54a80733be368074c63f007ac821a09040/merged major:0 minor:96 fsType:overlay blockSize:0}] Nov 03 21:45:54.713489 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.713037 6524 manager.go:217] Machine: {Timestamp:2026-11-03 21:45:54.712557895 +0000 UTC m=+0.064884690 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3134234 MemoryCapacity:33164324864 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec29333d281fee1b3e0524a91d19a59e SystemUUID:ec29333d-281f-ee1b-3e05-24a91d19a59e BootID:156f93a1-8311-43bd-a0df-1877bda020d8 Filesystems:[{Device:overlay_0-435 DeviceMajor:0 DeviceMinor:435 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-149 DeviceMajor:0 DeviceMinor:149 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-213 DeviceMajor:0 DeviceMinor:213 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bfd8df86-d3d0-4576-afd0-fa742a6a3aad/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:246 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:375 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volumes/kubernetes.io~projected/kube-api-access-mlgsq DeviceMajor:0 DeviceMinor:384 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/06948bda24b584813e431ecca9ab2c381f1466d883d2c21d599a46a87854cb4d/userdata/shm DeviceMajor:0 DeviceMinor:207 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/bfd8df86-d3d0-4576-afd0-fa742a6a3aad/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:238 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-299 DeviceMajor:0 DeviceMinor:299 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-441 DeviceMajor:0 DeviceMinor:441 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-159 DeviceMajor:0 DeviceMinor:159 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/93cbd093a41d6f8280f48bbb0d4906b6cf9bcedd415031f1e155400be7c35b12/userdata/shm DeviceMajor:0 DeviceMinor:61 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/da3b1b4886307f48cd0e54b04214c86fafa13ec3b4d3891c843639464ccd9707/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/32fd1d86-45ed-4a7d-894e-3990980bc237/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:203 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9d446131f627080dcfe0c7098bf0cc9b0e356964bd3cca77e96faf8eba808bb3/userdata/shm DeviceMajor:0 DeviceMinor:204 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0b349470d1bf258e06b2810cc311aff71f9653d22e5f6598ec3f038f6646a0bb/userdata/shm DeviceMajor:0 DeviceMinor:249 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/a2ccde5e-94a2-47ac-aff9-65248e4a6c92/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric DeviceMajor:0 DeviceMinor:264 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/76122905-15b4-4f4e-97b9-9f73a7675e36/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:49 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/484d94934d6014e3bd2e5fcfb68db6488c20d2d1ec8cefbc9075eeab26519f20/userdata/shm DeviceMajor:0 DeviceMinor:84 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/32fd1d86-45ed-4a7d-894e-3990980bc237/volumes/kubernetes.io~projected/kube-api-access-m8h9g DeviceMajor:0 DeviceMinor:205 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3eb641ca1e372ef33112a45ca00729446a14092b592b2c08fe59d58f33361bf4/userdata/shm DeviceMajor:0 DeviceMinor:285 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/43c10bc2bd2b94e42dbe0ea4ce382277116893404dc90d6a70e48102ec9fcbeb/userdata/shm DeviceMajor:0 DeviceMinor:431 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/866ddb5a-c929-40c1-911d-ca40c5405723/volumes/kubernetes.io~projected/kube-api-access-n5kf4 DeviceMajor:0 DeviceMinor:259 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ecfdcbf875981eb0a48d60cbc1003e95e865da3fd3fd6dec53c35397b6308e43/userdata/shm DeviceMajor:0 DeviceMinor:260 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:378 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/12d4118e93b819d9eb3d110e90fce86db0b42341bf9196e7600928553ea1077a/userdata/shm DeviceMajor:0 DeviceMinor:385 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volume-subpaths/web-config/config-reloader/2 DeviceMajor:0 DeviceMinor:383 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582164480 Type:vfs Inodes:1048576 HasInodes:true} {Device:/var/lib/kubelet/pods/9c970a61-3ebf-48c9-a05b-688accd0b18a/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:51 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-217 DeviceMajor:0 DeviceMinor:217 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bfd8df86-d3d0-4576-afd0-fa742a6a3aad/volumes/kubernetes.io~projected/kube-api-access-jl4cx DeviceMajor:0 DeviceMinor:239 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-294 DeviceMajor:0 DeviceMinor:294 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-345 DeviceMajor:0 DeviceMinor:345 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8783df8686233a4ddb7fe45426c063a89d5e82fbd6273021e42a43027efbd5de/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/a6cf0029-66be-4d86-ae8e-713d173a113e/volumes/kubernetes.io~projected/kube-api-access-zfkkl DeviceMajor:0 DeviceMinor:322 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls DeviceMajor:0 DeviceMinor:374 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ab4491eda7139fd31fdc7848d7f296f2c1f81b905d51c048f12a2ae3ea71da00/userdata/shm DeviceMajor:0 DeviceMinor:429 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5909793b2fc1d44e499b7ee12a8b1b1ff8d06a17a4790707c3eb3ab97c2259f0/userdata/shm DeviceMajor:0 DeviceMinor:209 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/a2ccde5e-94a2-47ac-aff9-65248e4a6c92/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy DeviceMajor:0 DeviceMinor:263 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-272 DeviceMajor:0 DeviceMinor:272 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-355 DeviceMajor:0 DeviceMinor:355 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/94aaa3ac-c7fc-4f87-878e-8ef79127488b/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:423 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-139 DeviceMajor:0 DeviceMinor:139 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volumes/kubernetes.io~secret/config DeviceMajor:0 DeviceMinor:377 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volumes/kubernetes.io~secret/secret-kube-rbac-proxy DeviceMajor:0 DeviceMinor:382 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-403 DeviceMajor:0 DeviceMinor:403 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/866ddb5a-c929-40c1-911d-ca40c5405723/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:257 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/a2ccde5e-94a2-47ac-aff9-65248e4a6c92/volume-subpaths/web-config/init-config-reloader/6 DeviceMajor:0 DeviceMinor:262 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-397 DeviceMajor:0 DeviceMinor:397 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/eb5f720e-f465-439f-ba32-4f595561f9f6/volumes/kubernetes.io~projected/kube-api-access-pgzcx DeviceMajor:0 DeviceMinor:59 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/5921d9cf-54a7-43bd-b697-c5ccf9f39135/volumes/kubernetes.io~projected/kube-api-access-t7m8x DeviceMajor:0 DeviceMinor:202 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-307 DeviceMajor:0 DeviceMinor:307 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/eb5f720e-f465-439f-ba32-4f595561f9f6/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:427 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c/volumes/kubernetes.io~projected/kube-api-access-csdhx DeviceMajor:0 DeviceMinor:48 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/219625cd-f1a4-4e51-90ee-58f0fc5939bb/volumes/kubernetes.io~projected/kube-api-access-kl5nr DeviceMajor:0 DeviceMinor:55 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/9c970a61-3ebf-48c9-a05b-688accd0b18a/volumes/kubernetes.io~projected/kube-api-access-9k8xx DeviceMajor:0 DeviceMinor:57 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-247 DeviceMajor:0 DeviceMinor:247 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-413 DeviceMajor:0 DeviceMinor:413 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-86 DeviceMajor:0 DeviceMinor:86 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/00c924c1-ac0d-49c8-8610-3b837fbb8277/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:279 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-389 DeviceMajor:0 DeviceMinor:389 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volume-subpaths/web-config/prometheus/15 DeviceMajor:0 DeviceMinor:383 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-146 DeviceMajor:0 DeviceMinor:146 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-236 DeviceMajor:0 DeviceMinor:236 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a2ccde5e-94a2-47ac-aff9-65248e4a6c92/volumes/kubernetes.io~projected/kube-api-access-s7g58 DeviceMajor:0 DeviceMinor:271 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-325 DeviceMajor:0 DeviceMinor:325 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582160384 Type:vfs Inodes:4048379 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/var/lib/kubelet/pods/76122905-15b4-4f4e-97b9-9f73a7675e36/volumes/kubernetes.io~projected/kube-api-access-tmr4b DeviceMajor:0 DeviceMinor:56 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/866ddb5a-c929-40c1-911d-ca40c5405723/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:258 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/a2ccde5e-94a2-47ac-aff9-65248e4a6c92/volume-subpaths/web-config/config-reloader/6 DeviceMajor:0 DeviceMinor:262 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volumes/kubernetes.io~secret/secret-metrics-client-certs DeviceMajor:0 DeviceMinor:376 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:383 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/9c970a61-3ebf-48c9-a05b-688accd0b18a/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:50 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-66 DeviceMajor:0 DeviceMinor:66 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-222 DeviceMajor:0 DeviceMinor:222 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-327 DeviceMajor:0 DeviceMinor:327 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:379 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volume-subpaths/web-config/init-config-reloader/2 DeviceMajor:0 DeviceMinor:383 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-408 DeviceMajor:0 DeviceMinor:408 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/00c924c1-ac0d-49c8-8610-3b837fbb8277/volumes/kubernetes.io~secret/secret-thanos-querier-tls DeviceMajor:0 DeviceMinor:282 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/5921d9cf-54a7-43bd-b697-c5ccf9f39135/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:45 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-225 DeviceMajor:0 DeviceMinor:225 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-255 DeviceMajor:0 DeviceMinor:255 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a2ccde5e-94a2-47ac-aff9-65248e4a6c92/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:269 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/a2ccde5e-94a2-47ac-aff9-65248e4a6c92/volume-subpaths/cluster-tls-config/alertmanager/10 DeviceMajor:0 DeviceMinor:268 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3bb48199a050488f952dbd7eefe0990c61e8638c924e084fbe0e9105710f7be6/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/00c924c1-ac0d-49c8-8610-3b837fbb8277/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules DeviceMajor:0 DeviceMinor:278 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/00c924c1-ac0d-49c8-8610-3b837fbb8277/volumes/kubernetes.io~projected/kube-api-access-phmrg DeviceMajor:0 DeviceMinor:284 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-301 DeviceMajor:0 DeviceMinor:301 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-395 DeviceMajor:0 DeviceMinor:395 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-449 DeviceMajor:0 DeviceMinor:449 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a2ccde5e-94a2-47ac-aff9-65248e4a6c92/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:262 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/106c43f9fcf1a1c3acc5d75e8010b45678a9a16af360e8ab411f4c897fd8dbc2/userdata/shm DeviceMajor:0 DeviceMinor:274 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/9f5c183e-d58a-4621-940c-04a7ae0bafc9/volumes/kubernetes.io~projected/kube-api-access-rg4r2 DeviceMajor:0 DeviceMinor:47 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/48d8b783-b602-47ef-9b9b-f2416d448908/volumes/kubernetes.io~projected/kube-api-access-k8fqs DeviceMajor:0 DeviceMinor:54 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/e652c3ec-d5ac-43f9-ade5-d3da32041cee/volumes/kubernetes.io~projected/kube-api-access-p7xhd DeviceMajor:0 DeviceMinor:201 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-418 DeviceMajor:0 DeviceMinor:418 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c244c1362b3ac28277d760dd45ee267608cffbbcd8811e1b3de0d2f5e23796df/userdata/shm DeviceMajor:0 DeviceMinor:60 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/76122905-15b4-4f4e-97b9-9f73a7675e36/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632865792 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/a2ccde5e-94a2-47ac-aff9-65248e4a6c92/volumes/kubernetes.io~secret/cluster-tls-config DeviceMajor:0 DeviceMinor:268 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-437 DeviceMajor:0 DeviceMinor:437 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/cae35dc92e36aaa91e303dd7f694cff5e2cc1a2b8830c0eda5a4496a101f574e/userdata/shm DeviceMajor:0 DeviceMinor:62 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/e652c3ec-d5ac-43f9-ade5-d3da32041cee/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:197 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-289 DeviceMajor:0 DeviceMinor:289 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a6cf0029-66be-4d86-ae8e-713d173a113e/volumes/kubernetes.io~secret/monitoring-plugin-cert DeviceMajor:0 DeviceMinor:318 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-439 DeviceMajor:0 DeviceMinor:439 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-230 DeviceMajor:0 DeviceMinor:230 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a2ccde5e-94a2-47ac-aff9-65248e4a6c92/volumes/kubernetes.io~secret/config-volume DeviceMajor:0 DeviceMinor:265 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/00c924c1-ac0d-49c8-8610-3b837fbb8277/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy DeviceMajor:0 DeviceMinor:281 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-329 DeviceMajor:0 DeviceMinor:329 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-334 DeviceMajor:0 DeviceMinor:334 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-365 DeviceMajor:0 DeviceMinor:365 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0db6d946-a1d8-4459-a883-6f93f7e1182a/volumes/kubernetes.io~projected/kube-api-access-6c6w8 DeviceMajor:0 DeviceMinor:58 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/14b5af6921fb45781a99bda05244046395650b5904b97226a33bfda38089f77f/userdata/shm DeviceMajor:0 DeviceMinor:77 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3cb26baa6890fd84328a69c267cd625706f3fd3c705d645908c5d46f151828b3/userdata/shm DeviceMajor:0 DeviceMinor:323 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-347 DeviceMajor:0 DeviceMinor:347 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/5d7e7316-1bd4-42cf-8519-544c1e2a4526/volumes/kubernetes.io~projected/kube-api-access-zh4kw DeviceMajor:0 DeviceMinor:428 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b251d8a015579cceb541952f1ebcb8e675394a3ad80016d966f745b49bbfb30c/userdata/shm DeviceMajor:0 DeviceMinor:433 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/a440580e-b0bd-49e2-a33d-576118e049ae/volumes/kubernetes.io~projected/kube-api-access-lqplq DeviceMajor:0 DeviceMinor:53 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/00c924c1-ac0d-49c8-8610-3b837fbb8277/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:280 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-387 DeviceMajor:0 DeviceMinor:387 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-454 DeviceMajor:0 DeviceMinor:454 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/811514f5-baff-4fc1-9e92-28f27de7b0b7/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:52 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:381 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-68 DeviceMajor:0 DeviceMinor:68 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3f1b9e13ddbf6ca5a0b73da16e9f34643e98cbe21681f3aa9c6c28106302b181/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-109 DeviceMajor:0 DeviceMinor:109 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-313 DeviceMajor:0 DeviceMinor:313 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632865792 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/00c924c1-ac0d-49c8-8610-3b837fbb8277/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics DeviceMajor:0 DeviceMinor:283 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/a2ccde5e-94a2-47ac-aff9-65248e4a6c92/volume-subpaths/web-config/alertmanager/9 DeviceMajor:0 DeviceMinor:262 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-336 DeviceMajor:0 DeviceMinor:336 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a2ccde5e-94a2-47ac-aff9-65248e4a6c92/volumes/kubernetes.io~secret/secret-alertmanager-main-tls DeviceMajor:0 DeviceMinor:270 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-276 DeviceMajor:0 DeviceMinor:276 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-287 DeviceMajor:0 DeviceMinor:287 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1b370a6c830e98f01be8c325b5aecac0f16f4ab08328435306f38e3b5b58aea7/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-360 DeviceMajor:0 DeviceMinor:360 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file DeviceMajor:0 DeviceMinor:370 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls DeviceMajor:0 DeviceMinor:380 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-443 DeviceMajor:0 DeviceMinor:443 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a2ccde5e-94a2-47ac-aff9-65248e4a6c92/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:266 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/a2ccde5e-94a2-47ac-aff9-65248e4a6c92/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:267 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:06948bda24b5848 MacAddress:16:86:48:67:a4:d9 Speed:10000 Mtu:8901} {Name:106c43f9fcf1a1c MacAddress:3e:05:b1:f4:9a:dd Speed:10000 Mtu:8901} {Name:12d4118e93b819d MacAddress:ea:6f:00:8a:bd:0b Speed:10000 Mtu:8901} {Name:3cb26baa6890fd8 MacAddress:aa:cc:9e:16:8a:5a Speed:10000 Mtu:8901} {Name:3eb641ca1e372ef MacAddress:5e:c8:28:ae:c5:43 Speed:10000 Mtu:8901} {Name:43c10bc2bd2b94e MacAddress:a2:50:fd:2b:13:2e Speed:10000 Mtu:8901} {Name:5909793b2fc1d44 MacAddress:fa:40:9c:6b:0e:cf Speed:10000 Mtu:8901} {Name:9d446131f627080 MacAddress:9a:eb:87:c3:d2:1e Speed:10000 Mtu:8901} {Name:ab4491eda7139fd MacAddress:8a:4b:9f:2b:12:ee Speed:10000 Mtu:8901} {Name:b251d8a015579cc MacAddress:ce:43:eb:1d:be:59 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:d2:0e:1b:c7 Speed:0 Mtu:9001} {Name:br-int MacAddress:1a:e7:8f:eb:0d:19 Speed:0 Mtu:8901} {Name:ecfdcbf875981eb MacAddress:1a:91:97:c3:2e:8f Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:d2:0e:1b:c7 Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:de:31:5e:8a:01:29 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:85:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:f2:c4:96:0d:9b:7c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164324864 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Nov 03 21:45:54.713716 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.713496 6524 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Nov 03 21:45:54.713716 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.713549 6524 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Nov 03 21:45:54.713877 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.713846 6524 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Nov 03 21:45:54.714010 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.713878 6524 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-7.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Nov 03 21:45:54.714050 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.714013 6524 topology_manager.go:143] "Creating topology manager with none policy" Nov 03 21:45:54.714050 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.714021 6524 container_manager_linux.go:311] "Creating device plugin manager" Nov 03 21:45:54.714050 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.714032 6524 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Nov 03 21:45:54.714141 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.714052 6524 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Nov 03 21:45:54.714175 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.714171 6524 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Nov 03 21:45:54.714357 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.714347 6524 state_mem.go:41] "Initialized" logger="CPUManager state memory" Nov 03 21:45:54.714471 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.714463 6524 server.go:1235] "Using root directory" path="/var/lib/kubelet" Nov 03 21:45:54.714505 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.714498 6524 kubelet.go:493] "Attempting to sync node with API server" Nov 03 21:45:54.714534 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.714513 6524 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Nov 03 21:45:54.714534 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.714523 6524 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Nov 03 21:45:54.714534 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.714531 6524 kubelet.go:405] "Adding apiserver pod source" Nov 03 21:45:54.714646 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.714539 6524 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Nov 03 21:45:54.715403 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.715374 6524 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Nov 03 21:45:54.715403 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.715386 6524 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Nov 03 21:45:54.715937 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.715918 6524 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Nov 03 21:45:54.719265 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.719249 6524 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Nov 03 21:45:54.719265 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.719263 6524 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Nov 03 21:45:54.719390 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.719368 6524 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem" Nov 03 21:45:54.719568 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.719556 6524 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Nov 03 21:45:54.719600 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.719586 6524 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Nov 03 21:45:54.719702 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.719692 6524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Nov 03 21:45:54.719736 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.719707 6524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Nov 03 21:45:54.719736 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.719715 6524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Nov 03 21:45:54.719736 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.719725 6524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Nov 03 21:45:54.719736 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.719734 6524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Nov 03 21:45:54.719853 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.719740 6524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Nov 03 21:45:54.719853 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.719746 6524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Nov 03 21:45:54.719853 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.719752 6524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Nov 03 21:45:54.719853 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.719758 6524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Nov 03 21:45:54.719853 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.719764 6524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Nov 03 21:45:54.719853 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.719773 6524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Nov 03 21:45:54.719853 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.719782 6524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Nov 03 21:45:54.719853 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.719823 6524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Nov 03 21:45:54.719853 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.719830 6524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Nov 03 21:45:54.723413 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.723390 6524 server.go:1263] "Started kubelet" Nov 03 21:45:54.723552 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.723443 6524 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Nov 03 21:45:54.724018 ip-10-0-137-7 systemd[1]: Started Kubernetes Kubelet. Nov 03 21:45:54.724774 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.724190 6524 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Nov 03 21:45:54.724774 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.724275 6524 server_v1.go:49] "podresources" method="list" useActivePods=true Nov 03 21:45:54.724774 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.724605 6524 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Nov 03 21:45:54.725839 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.725819 6524 server.go:317] "Adding debug handlers to kubelet server" Nov 03 21:45:54.731132 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.731095 6524 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Nov 03 21:45:54.731281 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.731150 6524 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Nov 03 21:45:54.731281 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.731207 6524 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-02 21:39:45 +0000 UTC" deadline="2028-03-29 18:52:57.924645369 +0000 UTC" Nov 03 21:45:54.731281 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.731232 6524 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12285h7m3.193417012s" Nov 03 21:45:54.731435 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.731309 6524 volume_manager.go:309] "The desired_state_of_world populator starts" Nov 03 21:45:54.731435 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.731321 6524 volume_manager.go:311] "Starting Kubelet Volume Manager" Nov 03 21:45:54.731867 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:54.731838 6524 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-137-7.us-east-2.compute.internal\" not found" Nov 03 21:45:54.731867 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.731844 6524 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Nov 03 21:45:54.734613 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.734551 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="00c924c1-ac0d-49c8-8610-3b837fbb8277" volumeName="kubernetes.io/secret/00c924c1-ac0d-49c8-8610-3b837fbb8277-secret-thanos-querier-kube-rbac-proxy-web" seLinuxMountContext="" Nov 03 21:45:54.734613 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.734606 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="00c924c1-ac0d-49c8-8610-3b837fbb8277" volumeName="kubernetes.io/secret/00c924c1-ac0d-49c8-8610-3b837fbb8277-secret-thanos-querier-tls" seLinuxMountContext="" Nov 03 21:45:54.734723 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.734622 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" volumeName="kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-k8s-rulefiles-2" seLinuxMountContext="" Nov 03 21:45:54.734723 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.734635 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="76122905-15b4-4f4e-97b9-9f73a7675e36" volumeName="kubernetes.io/projected/76122905-15b4-4f4e-97b9-9f73a7675e36-kube-api-access-tmr4b" seLinuxMountContext="" Nov 03 21:45:54.734723 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.734648 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" volumeName="kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-secret-alertmanager-kube-rbac-proxy" seLinuxMountContext="" Nov 03 21:45:54.734723 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.734661 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e652c3ec-d5ac-43f9-ade5-d3da32041cee" volumeName="kubernetes.io/projected/e652c3ec-d5ac-43f9-ade5-d3da32041cee-kube-api-access-p7xhd" seLinuxMountContext="" Nov 03 21:45:54.734723 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.734674 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="00c924c1-ac0d-49c8-8610-3b837fbb8277" volumeName="kubernetes.io/secret/00c924c1-ac0d-49c8-8610-3b837fbb8277-secret-thanos-querier-kube-rbac-proxy-rules" seLinuxMountContext="" Nov 03 21:45:54.734723 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.734686 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="219625cd-f1a4-4e51-90ee-58f0fc5939bb" volumeName="kubernetes.io/configmap/219625cd-f1a4-4e51-90ee-58f0fc5939bb-cni-binary-copy" seLinuxMountContext="" Nov 03 21:45:54.734723 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.734699 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5921d9cf-54a7-43bd-b697-c5ccf9f39135" volumeName="kubernetes.io/empty-dir/5921d9cf-54a7-43bd-b697-c5ccf9f39135-tmp-dir" seLinuxMountContext="" Nov 03 21:45:54.734723 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.734711 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="76122905-15b4-4f4e-97b9-9f73a7675e36" volumeName="kubernetes.io/configmap/76122905-15b4-4f4e-97b9-9f73a7675e36-env-overrides" seLinuxMountContext="" Nov 03 21:45:54.734723 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.734724 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="76122905-15b4-4f4e-97b9-9f73a7675e36" volumeName="kubernetes.io/secret/76122905-15b4-4f4e-97b9-9f73a7675e36-ovn-node-metrics-cert" seLinuxMountContext="" Nov 03 21:45:54.735027 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.734735 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="811514f5-baff-4fc1-9e92-28f27de7b0b7" volumeName="kubernetes.io/secret/811514f5-baff-4fc1-9e92-28f27de7b0b7-agent-certs" seLinuxMountContext="" Nov 03 21:45:54.735027 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.734747 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" volumeName="kubernetes.io/empty-dir/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-alertmanager-main-db" seLinuxMountContext="" Nov 03 21:45:54.735027 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.734758 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e652c3ec-d5ac-43f9-ade5-d3da32041cee" volumeName="kubernetes.io/secret/e652c3ec-d5ac-43f9-ade5-d3da32041cee-insights-runtime-extractor-tls" seLinuxMountContext="" Nov 03 21:45:54.735027 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.734772 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" volumeName="kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-configmap-serving-certs-ca-bundle" seLinuxMountContext="" Nov 03 21:45:54.735027 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.734783 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="866ddb5a-c929-40c1-911d-ca40c5405723" volumeName="kubernetes.io/projected/866ddb5a-c929-40c1-911d-ca40c5405723-kube-api-access-n5kf4" seLinuxMountContext="" Nov 03 21:45:54.735027 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.734796 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="866ddb5a-c929-40c1-911d-ca40c5405723" volumeName="kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-serving-cert" seLinuxMountContext="" Nov 03 21:45:54.735027 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.734808 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="94aaa3ac-c7fc-4f87-878e-8ef79127488b" volumeName="kubernetes.io/secret/94aaa3ac-c7fc-4f87-878e-8ef79127488b-original-pull-secret" seLinuxMountContext="" Nov 03 21:45:54.735027 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.734819 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9c970a61-3ebf-48c9-a05b-688accd0b18a" volumeName="kubernetes.io/empty-dir/9c970a61-3ebf-48c9-a05b-688accd0b18a-tmp" seLinuxMountContext="" Nov 03 21:45:54.735027 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.734836 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" volumeName="kubernetes.io/projected/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-tls-assets" seLinuxMountContext="" Nov 03 21:45:54.735027 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.734846 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eb5f720e-f465-439f-ba32-4f595561f9f6" volumeName="kubernetes.io/secret/eb5f720e-f465-439f-ba32-4f595561f9f6-metrics-certs" seLinuxMountContext="" Nov 03 21:45:54.735027 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.734859 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" volumeName="kubernetes.io/empty-dir/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-k8s-db" seLinuxMountContext="" Nov 03 21:45:54.735027 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.734873 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" volumeName="kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-prometheus-k8s-thanos-sidecar-tls" seLinuxMountContext="" Nov 03 21:45:54.735027 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.734886 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" volumeName="kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-kube-rbac-proxy" seLinuxMountContext="" Nov 03 21:45:54.735027 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.734896 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" volumeName="kubernetes.io/configmap/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-metrics-client-ca" seLinuxMountContext="" Nov 03 21:45:54.735027 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.734909 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="00c924c1-ac0d-49c8-8610-3b837fbb8277" volumeName="kubernetes.io/secret/00c924c1-ac0d-49c8-8610-3b837fbb8277-secret-thanos-querier-kube-rbac-proxy-metrics" seLinuxMountContext="" Nov 03 21:45:54.735027 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.734921 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" volumeName="kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-prometheus-k8s-kube-rbac-proxy-web" seLinuxMountContext="" Nov 03 21:45:54.735027 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.734933 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c" volumeName="kubernetes.io/projected/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c-kube-api-access-csdhx" seLinuxMountContext="" Nov 03 21:45:54.735027 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.734965 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9f5c183e-d58a-4621-940c-04a7ae0bafc9" volumeName="kubernetes.io/projected/9f5c183e-d58a-4621-940c-04a7ae0bafc9-kube-api-access-rg4r2" seLinuxMountContext="" Nov 03 21:45:54.735027 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.734980 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" volumeName="kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-secret-alertmanager-kube-rbac-proxy-metric" seLinuxMountContext="" Nov 03 21:45:54.735027 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735010 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" volumeName="kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-cluster-tls-config" seLinuxMountContext="" Nov 03 21:45:54.735027 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735025 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a440580e-b0bd-49e2-a33d-576118e049ae" volumeName="kubernetes.io/projected/a440580e-b0bd-49e2-a33d-576118e049ae-kube-api-access-lqplq" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735037 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bfd8df86-d3d0-4576-afd0-fa742a6a3aad" volumeName="kubernetes.io/secret/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-node-exporter-kube-rbac-proxy-config" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735060 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="00c924c1-ac0d-49c8-8610-3b837fbb8277" volumeName="kubernetes.io/configmap/00c924c1-ac0d-49c8-8610-3b837fbb8277-metrics-client-ca" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735077 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" volumeName="kubernetes.io/projected/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-kube-api-access-mlgsq" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735090 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" volumeName="kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-config" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735102 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="32fd1d86-45ed-4a7d-894e-3990980bc237" volumeName="kubernetes.io/secret/32fd1d86-45ed-4a7d-894e-3990980bc237-cert" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735113 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5921d9cf-54a7-43bd-b697-c5ccf9f39135" volumeName="kubernetes.io/configmap/5921d9cf-54a7-43bd-b697-c5ccf9f39135-config-volume" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735140 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" volumeName="kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-secret-alertmanager-main-tls" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735152 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bfd8df86-d3d0-4576-afd0-fa742a6a3aad" volumeName="kubernetes.io/projected/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-kube-api-access-jl4cx" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735165 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="866ddb5a-c929-40c1-911d-ca40c5405723" volumeName="kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-console-config" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735187 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" volumeName="kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-configmap-kubelet-serving-ca-bundle" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735198 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" volumeName="kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-prometheus-k8s-tls" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735211 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="32fd1d86-45ed-4a7d-894e-3990980bc237" volumeName="kubernetes.io/projected/32fd1d86-45ed-4a7d-894e-3990980bc237-kube-api-access-m8h9g" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735223 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5d7e7316-1bd4-42cf-8519-544c1e2a4526" volumeName="kubernetes.io/projected/5d7e7316-1bd4-42cf-8519-544c1e2a4526-kube-api-access-zh4kw" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735234 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c" volumeName="kubernetes.io/configmap/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c-cni-binary-copy" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735246 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9f5c183e-d58a-4621-940c-04a7ae0bafc9" volumeName="kubernetes.io/empty-dir/9f5c183e-d58a-4621-940c-04a7ae0bafc9-tmp-dir" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735271 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a440580e-b0bd-49e2-a33d-576118e049ae" volumeName="kubernetes.io/configmap/a440580e-b0bd-49e2-a33d-576118e049ae-serviceca" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735285 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" volumeName="kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-k8s-rulefiles-1" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735304 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5921d9cf-54a7-43bd-b697-c5ccf9f39135" volumeName="kubernetes.io/projected/5921d9cf-54a7-43bd-b697-c5ccf9f39135-kube-api-access-t7m8x" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735315 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="76122905-15b4-4f4e-97b9-9f73a7675e36" volumeName="kubernetes.io/configmap/76122905-15b4-4f4e-97b9-9f73a7675e36-ovnkube-script-lib" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735328 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eb5f720e-f465-439f-ba32-4f595561f9f6" volumeName="kubernetes.io/projected/eb5f720e-f465-439f-ba32-4f595561f9f6-kube-api-access-pgzcx" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735342 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="00c924c1-ac0d-49c8-8610-3b837fbb8277" volumeName="kubernetes.io/secret/00c924c1-ac0d-49c8-8610-3b837fbb8277-secret-grpc-tls" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735353 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5921d9cf-54a7-43bd-b697-c5ccf9f39135" volumeName="kubernetes.io/secret/5921d9cf-54a7-43bd-b697-c5ccf9f39135-metrics-tls" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735365 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0db6d946-a1d8-4459-a883-6f93f7e1182a" volumeName="kubernetes.io/projected/0db6d946-a1d8-4459-a883-6f93f7e1182a-kube-api-access-6c6w8" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735378 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" volumeName="kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-configmap-metrics-client-ca" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735400 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" volumeName="kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-thanos-prometheus-http-client-file" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735427 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="219625cd-f1a4-4e51-90ee-58f0fc5939bb" volumeName="kubernetes.io/configmap/219625cd-f1a4-4e51-90ee-58f0fc5939bb-multus-daemon-config" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735440 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9c970a61-3ebf-48c9-a05b-688accd0b18a" volumeName="kubernetes.io/empty-dir/9c970a61-3ebf-48c9-a05b-688accd0b18a-etc-tuned" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735451 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" volumeName="kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-secret-alertmanager-kube-rbac-proxy-web" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735463 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e652c3ec-d5ac-43f9-ade5-d3da32041cee" volumeName="kubernetes.io/empty-dir/e652c3ec-d5ac-43f9-ade5-d3da32041cee-data-volume" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735482 6524 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735488 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" volumeName="kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-k8s-rulefiles-0" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735522 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" volumeName="kubernetes.io/projected/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-tls-assets" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735542 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="76122905-15b4-4f4e-97b9-9f73a7675e36" volumeName="kubernetes.io/configmap/76122905-15b4-4f4e-97b9-9f73a7675e36-ovnkube-config" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735557 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" volumeName="kubernetes.io/configmap/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-alertmanager-trusted-ca-bundle" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735572 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bfd8df86-d3d0-4576-afd0-fa742a6a3aad" volumeName="kubernetes.io/configmap/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-node-exporter-accelerators-collector-config" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735586 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="00c924c1-ac0d-49c8-8610-3b837fbb8277" volumeName="kubernetes.io/secret/00c924c1-ac0d-49c8-8610-3b837fbb8277-secret-thanos-querier-kube-rbac-proxy" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735598 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0db6d946-a1d8-4459-a883-6f93f7e1182a" volumeName="kubernetes.io/configmap/0db6d946-a1d8-4459-a883-6f93f7e1182a-iptables-alerter-script" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735615 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" volumeName="kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-trusted-ca-bundle" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735631 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="219625cd-f1a4-4e51-90ee-58f0fc5939bb" volumeName="kubernetes.io/projected/219625cd-f1a4-4e51-90ee-58f0fc5939bb-kube-api-access-kl5nr" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735644 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="811514f5-baff-4fc1-9e92-28f27de7b0b7" volumeName="kubernetes.io/configmap/811514f5-baff-4fc1-9e92-28f27de7b0b7-konnectivity-ca" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735656 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="866ddb5a-c929-40c1-911d-ca40c5405723" volumeName="kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-service-ca" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735668 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="866ddb5a-c929-40c1-911d-ca40c5405723" volumeName="kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-oauth-config" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735683 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a6cf0029-66be-4d86-ae8e-713d173a113e" volumeName="kubernetes.io/projected/a6cf0029-66be-4d86-ae8e-713d173a113e-kube-api-access-zfkkl" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735696 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c" volumeName="kubernetes.io/configmap/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c-whereabouts-flatfile-configmap" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735710 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="866ddb5a-c929-40c1-911d-ca40c5405723" volumeName="kubernetes.io/empty-dir/866ddb5a-c929-40c1-911d-ca40c5405723-tmp" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735722 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" volumeName="kubernetes.io/projected/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-kube-api-access-s7g58" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735734 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bfd8df86-d3d0-4576-afd0-fa742a6a3aad" volumeName="kubernetes.io/empty-dir/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-node-exporter-textfile" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735750 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="00c924c1-ac0d-49c8-8610-3b837fbb8277" volumeName="kubernetes.io/empty-dir/00c924c1-ac0d-49c8-8610-3b837fbb8277-tmp" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735762 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" volumeName="kubernetes.io/empty-dir/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-config-out" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735776 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" volumeName="kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-grpc-tls" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735787 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="48d8b783-b602-47ef-9b9b-f2416d448908" volumeName="kubernetes.io/projected/48d8b783-b602-47ef-9b9b-f2416d448908-kube-api-access-k8fqs" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735802 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c" volumeName="kubernetes.io/configmap/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c-cni-sysctl-allowlist" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735816 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9c970a61-3ebf-48c9-a05b-688accd0b18a" volumeName="kubernetes.io/projected/9c970a61-3ebf-48c9-a05b-688accd0b18a-kube-api-access-9k8xx" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735859 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" volumeName="kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-config-volume" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735877 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a6cf0029-66be-4d86-ae8e-713d173a113e" volumeName="kubernetes.io/secret/a6cf0029-66be-4d86-ae8e-713d173a113e-monitoring-plugin-cert" seLinuxMountContext="" Nov 03 21:45:54.735878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735892 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="866ddb5a-c929-40c1-911d-ca40c5405723" volumeName="kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-oauth-serving-cert" seLinuxMountContext="" Nov 03 21:45:54.737979 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735953 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" volumeName="kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-web-config" seLinuxMountContext="" Nov 03 21:45:54.737979 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735974 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bfd8df86-d3d0-4576-afd0-fa742a6a3aad" volumeName="kubernetes.io/configmap/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-metrics-client-ca" seLinuxMountContext="" Nov 03 21:45:54.737979 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.735990 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bfd8df86-d3d0-4576-afd0-fa742a6a3aad" volumeName="kubernetes.io/secret/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-node-exporter-tls" seLinuxMountContext="" Nov 03 21:45:54.737979 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.736002 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e652c3ec-d5ac-43f9-ade5-d3da32041cee" volumeName="kubernetes.io/configmap/e652c3ec-d5ac-43f9-ade5-d3da32041cee-kube-rbac-proxy-cm" seLinuxMountContext="" Nov 03 21:45:54.737979 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.736015 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="00c924c1-ac0d-49c8-8610-3b837fbb8277" volumeName="kubernetes.io/projected/00c924c1-ac0d-49c8-8610-3b837fbb8277-kube-api-access-phmrg" seLinuxMountContext="" Nov 03 21:45:54.737979 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.736027 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" volumeName="kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-metrics-client-certs" seLinuxMountContext="" Nov 03 21:45:54.737979 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.736040 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" volumeName="kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-web-config" seLinuxMountContext="" Nov 03 21:45:54.737979 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.736054 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="866ddb5a-c929-40c1-911d-ca40c5405723" volumeName="kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-trusted-ca-bundle" seLinuxMountContext="" Nov 03 21:45:54.737979 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.736066 6524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" volumeName="kubernetes.io/empty-dir/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-config-out" seLinuxMountContext="" Nov 03 21:45:54.737979 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.736078 6524 reconstruct.go:97] "Volume reconstruction finished" Nov 03 21:45:54.737979 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.736085 6524 reconciler.go:29] "Reconciler: start to sync state" Nov 03 21:45:54.737979 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.736282 6524 factory.go:153] Registering CRI-O factory Nov 03 21:45:54.737979 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.736298 6524 factory.go:223] Registration of the crio container factory successfully Nov 03 21:45:54.737979 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.736359 6524 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Nov 03 21:45:54.737979 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.736368 6524 factory.go:55] Registering systemd factory Nov 03 21:45:54.737979 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.736376 6524 factory.go:223] Registration of the systemd container factory successfully Nov 03 21:45:54.737979 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.736393 6524 factory.go:103] Registering Raw factory Nov 03 21:45:54.737979 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.736408 6524 manager.go:1199] Started watching for new ooms in manager Nov 03 21:45:54.737979 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.736763 6524 manager.go:319] Starting recovery of all containers Nov 03 21:45:54.738855 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.738087 6524 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Nov 03 21:45:54.741054 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.741030 6524 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Nov 03 21:45:54.745215 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.742405 6524 reconstruct.go:205] "DevicePaths of reconstructed volumes updated" Nov 03 21:45:54.745215 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.743665 6524 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Nov 03 21:45:54.760754 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.760723 6524 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Nov 03 21:45:54.760754 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.760751 6524 status_manager.go:249] "Starting to sync pod status with apiserver" Nov 03 21:45:54.760955 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.760770 6524 kubelet.go:2521] "Starting kubelet main sync loop" Nov 03 21:45:54.760955 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.760789 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x26mj_76122905-15b4-4f4e-97b9-9f73a7675e36/ovn-acl-logging/0.log" Nov 03 21:45:54.760955 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:54.760820 6524 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Nov 03 21:45:54.761425 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.761408 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x26mj_76122905-15b4-4f4e-97b9-9f73a7675e36/ovn-acl-logging/0.log" Nov 03 21:45:54.765688 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.765662 6524 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Nov 03 21:45:54.777245 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.777214 6524 generic.go:356] "Generic (PLEG): container finished" podID="bfd8df86-d3d0-4576-afd0-fa742a6a3aad" containerID="4d61365efa6dfaf176e3e5455b1ab86a931077e22613ce4fc29799f8e459d000" exitCode=0 Nov 03 21:45:54.783910 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.783878 6524 generic.go:356] "Generic (PLEG): container finished" podID="58ba030777ca53e38084de26650a107e" containerID="cfde88035187ebb9aabafdac7ec45b9549ab3ac9180a83951ebcf5b331611c03" exitCode=0 Nov 03 21:45:54.790799 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.790766 6524 generic.go:356] "Generic (PLEG): container finished" podID="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" containerID="14b9495cfb235a428a51ba26e6ccea26a7e4b5c67bbe3b4f43a6d924eff2decf" exitCode=0 Nov 03 21:45:54.796011 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.795987 6524 generic.go:356] "Generic (PLEG): container finished" podID="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" containerID="1d7e1a070f9e0542150676350d5dbd14e69951f115872bce24a88b74c9fb9a68" exitCode=0 Nov 03 21:45:54.806600 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.806568 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x26mj_76122905-15b4-4f4e-97b9-9f73a7675e36/ovn-acl-logging/0.log" Nov 03 21:45:54.806982 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.806956 6524 generic.go:356] "Generic (PLEG): container finished" podID="76122905-15b4-4f4e-97b9-9f73a7675e36" containerID="c4b707b05dcedd7bb732861345630a8373dd963050b73d818d0d6e5b79c426e3" exitCode=1 Nov 03 21:45:54.810815 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.810789 6524 generic.go:356] "Generic (PLEG): container finished" podID="62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c" containerID="325011f326827b19ff920f51179dce33712de13818285eb3f64afba0b09d0761" exitCode=0 Nov 03 21:45:54.810815 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.810815 6524 generic.go:356] "Generic (PLEG): container finished" podID="62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c" containerID="da24cd5354d6f06b2fef4b3e4d1d5dffc5044edfc9cd86d18849ca9e9ae7d796" exitCode=0 Nov 03 21:45:54.810948 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.810824 6524 generic.go:356] "Generic (PLEG): container finished" podID="62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c" containerID="7d62af7781c487f815f911f6ad53df53abd463176f29148e201de910ee5a05bc" exitCode=0 Nov 03 21:45:54.810948 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.810831 6524 generic.go:356] "Generic (PLEG): container finished" podID="62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c" containerID="426b25f1f3bca20c5cb8f0743a5754565d6364332133f91c634b44f2a5f08a4c" exitCode=0 Nov 03 21:45:54.810948 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.810840 6524 generic.go:356] "Generic (PLEG): container finished" podID="62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c" containerID="bcfab2b6ff71f13e137f3d54a87305ef838d31642eb49639040928e01b6724c0" exitCode=0 Nov 03 21:45:54.810948 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.810846 6524 generic.go:356] "Generic (PLEG): container finished" podID="62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c" containerID="32e5e852a07ab99b25cf2390a0d6f5516d694632654f82058ab02ed363be305b" exitCode=0 Nov 03 21:45:54.848981 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.848910 6524 manager.go:324] Recovery completed Nov 03 21:45:54.861542 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:54.861516 6524 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet" Nov 03 21:45:54.873224 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.873199 6524 cpu_manager.go:226] "Starting" policy="none" Nov 03 21:45:54.873224 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.873216 6524 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Nov 03 21:45:54.873402 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.873235 6524 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Nov 03 21:45:54.873402 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.873364 6524 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet="" Nov 03 21:45:54.873402 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.873374 6524 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={} Nov 03 21:45:54.873402 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.873391 6524 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint" Nov 03 21:45:54.873402 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.873396 6524 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet="" Nov 03 21:45:54.873402 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.873401 6524 policy_none.go:50] "Start" Nov 03 21:45:54.873601 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.873410 6524 memory_manager.go:187] "Starting memorymanager" policy="None" Nov 03 21:45:54.873601 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.873419 6524 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Nov 03 21:45:54.873601 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.873537 6524 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint" Nov 03 21:45:54.873601 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.873544 6524 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint" Nov 03 21:45:54.873601 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.873550 6524 policy_none.go:44] "Start" Nov 03 21:45:54.878220 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.878192 6524 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Nov 03 21:45:54.880822 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.880807 6524 manager.go:341] "Starting Device Plugin manager" Nov 03 21:45:54.881291 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:54.880831 6524 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Nov 03 21:45:54.881339 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.881293 6524 server.go:87] "Starting device plugin registration server" Nov 03 21:45:54.881554 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.881544 6524 eviction_manager.go:194] "Eviction manager: starting control loop" Nov 03 21:45:54.881592 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.881555 6524 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Nov 03 21:45:54.881668 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.881654 6524 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Nov 03 21:45:54.881770 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.881744 6524 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Nov 03 21:45:54.881822 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.881783 6524 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Nov 03 21:45:54.881822 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.881790 6524 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Nov 03 21:45:54.881904 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.881827 6524 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-03T21:45:54.881769081Z","UUID":"4092169b-fd05-494d-81b4-74f89453dee5","Handler":null,"Name":"","Endpoint":""} Nov 03 21:45:54.883438 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:54.883417 6524 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Nov 03 21:45:54.883701 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.883682 6524 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Nov 03 21:45:54.883811 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.883718 6524 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Nov 03 21:45:54.982066 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.982027 6524 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 03 21:45:54.984062 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.984040 6524 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-7.us-east-2.compute.internal" event="NodeHasSufficientMemory" Nov 03 21:45:54.984205 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.984077 6524 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-7.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Nov 03 21:45:54.984205 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.984092 6524 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-7.us-east-2.compute.internal" event="NodeHasSufficientPID" Nov 03 21:45:54.984205 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.984155 6524 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-137-7.us-east-2.compute.internal" Nov 03 21:45:54.998605 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.998572 6524 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-137-7.us-east-2.compute.internal" Nov 03 21:45:54.998723 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:54.998652 6524 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-137-7.us-east-2.compute.internal" Nov 03 21:45:55.062025 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.061950 6524 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-7.us-east-2.compute.internal"] Nov 03 21:45:55.062700 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.062653 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal" event={"ID":"58ba030777ca53e38084de26650a107e","Type":"ContainerStarted","Data":"b48be4756b3a13ddbd10852a623be839e5ef4bd3482585882307a1970c21e124"} Nov 03 21:45:55.062816 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.062707 6524 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal" Nov 03 21:45:55.062816 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.062712 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal" event={"ID":"58ba030777ca53e38084de26650a107e","Type":"ContainerDied","Data":"cfde88035187ebb9aabafdac7ec45b9549ab3ac9180a83951ebcf5b331611c03"} Nov 03 21:45:55.062816 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.062735 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal" event={"ID":"58ba030777ca53e38084de26650a107e","Type":"ContainerStarted","Data":"cae35dc92e36aaa91e303dd7f694cff5e2cc1a2b8830c0eda5a4496a101f574e"} Nov 03 21:45:55.062816 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.062763 6524 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-7.us-east-2.compute.internal" Nov 03 21:45:55.062970 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.062836 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-7.us-east-2.compute.internal" event={"ID":"70b8abbfbbbc47301b7da80ca6b44429","Type":"ContainerStarted","Data":"78d9df0a1516cc2535f73127c8f004dcb94dbd2eafea2a5bb9d643361b6d2428"} Nov 03 21:45:55.062970 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.062856 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-7.us-east-2.compute.internal" event={"ID":"70b8abbfbbbc47301b7da80ca6b44429","Type":"ContainerStarted","Data":"c244c1362b3ac28277d760dd45ee267608cffbbcd8811e1b3de0d2f5e23796df"} Nov 03 21:45:55.077373 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.077349 6524 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 03 21:45:55.077455 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:55.077393 6524 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-137-7.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-7.us-east-2.compute.internal" Nov 03 21:45:55.077815 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.077800 6524 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 03 21:45:55.077851 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:55.077838 6524 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal" Nov 03 21:45:55.144272 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.144173 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/58ba030777ca53e38084de26650a107e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal\" (UID: \"58ba030777ca53e38084de26650a107e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal" Nov 03 21:45:55.144272 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.144213 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/58ba030777ca53e38084de26650a107e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal\" (UID: \"58ba030777ca53e38084de26650a107e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal" Nov 03 21:45:55.144272 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.144243 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/70b8abbfbbbc47301b7da80ca6b44429-config\") pod \"kube-apiserver-proxy-ip-10-0-137-7.us-east-2.compute.internal\" (UID: \"70b8abbfbbbc47301b7da80ca6b44429\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-7.us-east-2.compute.internal" Nov 03 21:45:55.244797 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.244751 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/58ba030777ca53e38084de26650a107e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal\" (UID: \"58ba030777ca53e38084de26650a107e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal" Nov 03 21:45:55.245016 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.244805 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/58ba030777ca53e38084de26650a107e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal\" (UID: \"58ba030777ca53e38084de26650a107e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal" Nov 03 21:45:55.245016 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.244843 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/70b8abbfbbbc47301b7da80ca6b44429-config\") pod \"kube-apiserver-proxy-ip-10-0-137-7.us-east-2.compute.internal\" (UID: \"70b8abbfbbbc47301b7da80ca6b44429\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-7.us-east-2.compute.internal" Nov 03 21:45:55.245016 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.244771 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/58ba030777ca53e38084de26650a107e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal\" (UID: \"58ba030777ca53e38084de26650a107e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal" Nov 03 21:45:55.245016 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.244886 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/70b8abbfbbbc47301b7da80ca6b44429-config\") pod \"kube-apiserver-proxy-ip-10-0-137-7.us-east-2.compute.internal\" (UID: \"70b8abbfbbbc47301b7da80ca6b44429\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-7.us-east-2.compute.internal" Nov 03 21:45:55.245016 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.244925 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/58ba030777ca53e38084de26650a107e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal\" (UID: \"58ba030777ca53e38084de26650a107e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal" Nov 03 21:45:55.715584 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.715551 6524 apiserver.go:52] "Watching apiserver" Nov 03 21:45:55.730897 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.730860 6524 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Nov 03 21:45:55.731582 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.731550 6524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-86cf959768-tf4fd","openshift-monitoring/prometheus-k8s-0","openshift-cluster-node-tuning-operator/tuned-xw7xr","openshift-image-registry/node-ca-nz4zw","openshift-multus/multus-mbksx","openshift-network-operator/iptables-alerter-nkngm","kube-system/kube-apiserver-proxy-ip-10-0-137-7.us-east-2.compute.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal","openshift-network-diagnostics/network-check-target-zrbsd","openshift-ingress-canary/ingress-canary-77wdg","openshift-dns/node-resolver-9pctw","kube-system/konnectivity-agent-v98zn","openshift-monitoring/monitoring-plugin-77f4f76594-9cnqh","openshift-monitoring/node-exporter-jdbs4","openshift-insights/insights-runtime-extractor-285hg","openshift-ovn-kubernetes/ovnkube-node-x26mj","openshift-monitoring/thanos-querier-54674f497-2g5ct","openshift-dns/dns-default-x2r5s","openshift-multus/multus-additional-cni-plugins-4cz4g","openshift-multus/network-metrics-daemon-knt57","openshift-monitoring/alertmanager-main-0","kube-system/global-pull-secret-syncer-jq6hj","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c"] Nov 03 21:45:55.737944 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.736888 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-txfgg\"" Nov 03 21:45:55.737944 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.737015 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-jbpmc\"" Nov 03 21:45:55.737944 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.737159 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-p72km\"" Nov 03 21:45:55.737944 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.737362 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-bc4mv\"" Nov 03 21:45:55.737944 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.737710 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Nov 03 21:45:55.737944 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.737799 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Nov 03 21:45:55.737944 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.737811 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Nov 03 21:45:55.737944 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.737923 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Nov 03 21:45:55.738433 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.737973 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Nov 03 21:45:55.738433 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.737975 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Nov 03 21:45:55.738433 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.738329 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Nov 03 21:45:55.738433 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.738378 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Nov 03 21:45:55.738691 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.738669 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-fc4bx\"" Nov 03 21:45:55.738820 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.738718 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Nov 03 21:45:55.738820 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.738670 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Nov 03 21:45:55.738820 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.738778 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Nov 03 21:45:55.738952 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.738912 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-rdd92\"" Nov 03 21:45:55.739262 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.739246 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Nov 03 21:45:55.739358 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.739274 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Nov 03 21:45:55.739358 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.739273 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Nov 03 21:45:55.739454 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.739368 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Nov 03 21:45:55.739454 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.739415 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Nov 03 21:45:55.739573 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.739539 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Nov 03 21:45:55.739685 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.739578 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Nov 03 21:45:55.739685 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.739577 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-lqk2r\"" Nov 03 21:45:55.739852 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.739716 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Nov 03 21:45:55.739852 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.739737 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Nov 03 21:45:55.740099 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.740080 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Nov 03 21:45:55.740229 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.740147 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-hgwjm\"" Nov 03 21:45:55.741499 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.741479 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-t98zr\"" Nov 03 21:45:55.741585 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.741569 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Nov 03 21:45:55.741634 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.741602 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-5xcnq\"" Nov 03 21:45:55.741672 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.741631 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Nov 03 21:45:55.742271 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.742248 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Nov 03 21:45:55.742368 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.742355 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Nov 03 21:45:55.742428 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.742418 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-lbb62\"" Nov 03 21:45:55.743644 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.743628 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Nov 03 21:45:55.743980 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.743963 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-52n7c\"" Nov 03 21:45:55.744058 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.744022 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Nov 03 21:45:55.744058 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.744040 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Nov 03 21:45:55.744340 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.744325 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Nov 03 21:45:55.744462 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.744446 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Nov 03 21:45:55.745534 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.745505 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Nov 03 21:45:55.746013 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.745991 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Nov 03 21:45:55.746114 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.746031 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Nov 03 21:45:55.746198 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.746157 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Nov 03 21:45:55.746287 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.746262 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9f5c183e-d58a-4621-940c-04a7ae0bafc9-hosts-file\") pod \"node-resolver-9pctw\" (UID: \"9f5c183e-d58a-4621-940c-04a7ae0bafc9\") " pod="openshift-dns/node-resolver-9pctw" Nov 03 21:45:55.746389 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.746300 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f5c183e-d58a-4621-940c-04a7ae0bafc9-tmp-dir\") pod \"node-resolver-9pctw\" (UID: \"9f5c183e-d58a-4621-940c-04a7ae0bafc9\") " pod="openshift-dns/node-resolver-9pctw" Nov 03 21:45:55.746389 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.746353 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg4r2\" (UniqueName: \"kubernetes.io/projected/9f5c183e-d58a-4621-940c-04a7ae0bafc9-kube-api-access-rg4r2\") pod \"node-resolver-9pctw\" (UID: \"9f5c183e-d58a-4621-940c-04a7ae0bafc9\") " pod="openshift-dns/node-resolver-9pctw" Nov 03 21:45:55.746502 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.746399 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh4kw\" (UniqueName: \"kubernetes.io/projected/5d7e7316-1bd4-42cf-8519-544c1e2a4526-kube-api-access-zh4kw\") pod \"network-check-target-zrbsd\" (UID: \"5d7e7316-1bd4-42cf-8519-544c1e2a4526\") " pod="openshift-network-diagnostics/network-check-target-zrbsd" Nov 03 21:45:55.746502 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.746426 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f5c183e-d58a-4621-940c-04a7ae0bafc9-tmp-dir\") pod \"node-resolver-9pctw\" (UID: \"9f5c183e-d58a-4621-940c-04a7ae0bafc9\") " pod="openshift-dns/node-resolver-9pctw" Nov 03 21:45:55.747350 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.747318 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Nov 03 21:45:55.747434 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.747390 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Nov 03 21:45:55.747489 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.747457 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Nov 03 21:45:55.747559 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.747544 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Nov 03 21:45:55.747772 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.747753 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Nov 03 21:45:55.747932 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.747915 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Nov 03 21:45:55.748003 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.747962 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Nov 03 21:45:55.748003 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.747964 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-s5f4x\"" Nov 03 21:45:55.748108 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.748097 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Nov 03 21:45:55.748192 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.748181 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Nov 03 21:45:55.748237 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.748208 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-7bvj7\"" Nov 03 21:45:55.748478 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.748469 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-wc5kn\"" Nov 03 21:45:55.748585 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.748573 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Nov 03 21:45:55.748669 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.748654 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Nov 03 21:45:55.748669 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.748667 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-92tpk\"" Nov 03 21:45:55.748752 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.748684 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Nov 03 21:45:55.748752 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.748714 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Nov 03 21:45:55.749808 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.749795 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Nov 03 21:45:55.757164 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.757138 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg4r2\" (UniqueName: \"kubernetes.io/projected/9f5c183e-d58a-4621-940c-04a7ae0bafc9-kube-api-access-rg4r2\") pod \"node-resolver-9pctw\" (UID: \"9f5c183e-d58a-4621-940c-04a7ae0bafc9\") " pod="openshift-dns/node-resolver-9pctw" Nov 03 21:45:55.757294 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.757277 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zh4kw\" (UniqueName: \"kubernetes.io/projected/5d7e7316-1bd4-42cf-8519-544c1e2a4526-kube-api-access-zh4kw\") pod \"network-check-target-zrbsd\" (UID: \"5d7e7316-1bd4-42cf-8519-544c1e2a4526\") " pod="openshift-network-diagnostics/network-check-target-zrbsd" Nov 03 21:45:55.827983 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.827952 6524 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-7.us-east-2.compute.internal" Nov 03 21:45:55.828161 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.828055 6524 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal" Nov 03 21:45:55.835749 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.835003 6524 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Nov 03 21:45:55.839769 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.839737 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Nov 03 21:45:55.839769 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.839748 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Nov 03 21:45:55.839963 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.839852 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Nov 03 21:45:55.839963 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.839867 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Nov 03 21:45:55.839963 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.839895 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-8vqcv\"" Nov 03 21:45:55.839963 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.839897 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Nov 03 21:45:55.839963 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.839928 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Nov 03 21:45:55.839963 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.839941 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Nov 03 21:45:55.839963 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.839957 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Nov 03 21:45:55.839963 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.839964 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Nov 03 21:45:55.840236 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.839964 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Nov 03 21:45:55.840236 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.839984 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Nov 03 21:45:55.840236 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.839894 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Nov 03 21:45:55.840236 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.840000 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Nov 03 21:45:55.840736 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.840717 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Nov 03 21:45:55.840823 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.840788 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Nov 03 21:45:55.840932 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.840911 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Nov 03 21:45:55.840932 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.840912 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Nov 03 21:45:55.841130 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.840911 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Nov 03 21:45:55.841130 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.840946 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Nov 03 21:45:55.841130 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.840952 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Nov 03 21:45:55.841130 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.840950 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Nov 03 21:45:55.841130 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.840956 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Nov 03 21:45:55.841130 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.840985 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-6hj4n\"" Nov 03 21:45:55.841130 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.841003 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-52l5qoid38f49\"" Nov 03 21:45:55.841130 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.841035 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Nov 03 21:45:55.842777 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.842551 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-c54pc\"" Nov 03 21:45:55.842777 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.842560 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-khpcw\"" Nov 03 21:45:55.842777 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.842609 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-5s11ulothan92\"" Nov 03 21:45:55.842777 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.842569 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Nov 03 21:45:55.842777 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.842627 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Nov 03 21:45:55.847333 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.847315 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-systemd-units\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.847432 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.847342 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/76122905-15b4-4f4e-97b9-9f73a7675e36-env-overrides\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.847432 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.847369 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c-system-cni-dir\") pod \"multus-additional-cni-plugins-4cz4g\" (UID: \"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c\") " pod="openshift-multus/multus-additional-cni-plugins-4cz4g" Nov 03 21:45:55.847432 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.847392 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/48d8b783-b602-47ef-9b9b-f2416d448908-registration-dir\") pod \"aws-ebs-csi-driver-node-wvd2c\" (UID: \"48d8b783-b602-47ef-9b9b-f2416d448908\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" Nov 03 21:45:55.847568 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.847463 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-web-config\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:55.847568 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.847536 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/866ddb5a-c929-40c1-911d-ca40c5405723-tmp\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:55.847654 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.847583 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/76122905-15b4-4f4e-97b9-9f73a7675e36-env-overrides\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.847654 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.847633 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqplq\" (UniqueName: \"kubernetes.io/projected/a440580e-b0bd-49e2-a33d-576118e049ae-kube-api-access-lqplq\") pod \"node-ca-nz4zw\" (UID: \"a440580e-b0bd-49e2-a33d-576118e049ae\") " pod="openshift-image-registry/node-ca-nz4zw" Nov 03 21:45:55.847762 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.847666 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-etc-kubernetes\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.847762 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.847676 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/866ddb5a-c929-40c1-911d-ca40c5405723-tmp\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:55.847762 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.847692 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c-cni-binary-copy\") pod \"multus-additional-cni-plugins-4cz4g\" (UID: \"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c\") " pod="openshift-multus/multus-additional-cni-plugins-4cz4g" Nov 03 21:45:55.847762 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.847717 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4cz4g\" (UID: \"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c\") " pod="openshift-multus/multus-additional-cni-plugins-4cz4g" Nov 03 21:45:55.847762 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.847744 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:55.848001 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.847771 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:55.848001 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.847795 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-oauth-serving-cert\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:55.848001 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.847807 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-web-config\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:55.848001 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.847816 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-cnibin\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.848001 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.847861 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-hostroot\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.848001 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.847885 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e652c3ec-d5ac-43f9-ade5-d3da32041cee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-285hg\" (UID: \"e652c3ec-d5ac-43f9-ade5-d3da32041cee\") " pod="openshift-insights/insights-runtime-extractor-285hg" Nov 03 21:45:55.848001 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.847908 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5921d9cf-54a7-43bd-b697-c5ccf9f39135-tmp-dir\") pod \"dns-default-x2r5s\" (UID: \"5921d9cf-54a7-43bd-b697-c5ccf9f39135\") " pod="openshift-dns/dns-default-x2r5s" Nov 03 21:45:55.848001 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.847933 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c-cni-binary-copy\") pod \"multus-additional-cni-plugins-4cz4g\" (UID: \"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c\") " pod="openshift-multus/multus-additional-cni-plugins-4cz4g" Nov 03 21:45:55.848001 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.847956 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7m8x\" (UniqueName: \"kubernetes.io/projected/5921d9cf-54a7-43bd-b697-c5ccf9f39135-kube-api-access-t7m8x\") pod \"dns-default-x2r5s\" (UID: \"5921d9cf-54a7-43bd-b697-c5ccf9f39135\") " pod="openshift-dns/dns-default-x2r5s" Nov 03 21:45:55.848001 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.847991 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-root\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:55.848470 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.848006 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:55.848470 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.847994 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5921d9cf-54a7-43bd-b697-c5ccf9f39135-tmp-dir\") pod \"dns-default-x2r5s\" (UID: \"5921d9cf-54a7-43bd-b697-c5ccf9f39135\") " pod="openshift-dns/dns-default-x2r5s" Nov 03 21:45:55.848470 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:55.848025 6524 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered Nov 03 21:45:55.848470 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.848041 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4cz4g\" (UID: \"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c\") " pod="openshift-multus/multus-additional-cni-plugins-4cz4g" Nov 03 21:45:55.848470 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.848046 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:55.848470 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.848071 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/76122905-15b4-4f4e-97b9-9f73a7675e36-ovn-node-metrics-cert\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.848470 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.848094 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8h9g\" (UniqueName: \"kubernetes.io/projected/32fd1d86-45ed-4a7d-894e-3990980bc237-kube-api-access-m8h9g\") pod \"ingress-canary-77wdg\" (UID: \"32fd1d86-45ed-4a7d-894e-3990980bc237\") " pod="openshift-ingress-canary/ingress-canary-77wdg" Nov 03 21:45:55.848470 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:55.848116 6524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-oauth-serving-cert podName:866ddb5a-c929-40c1-911d-ca40c5405723 nodeName:}" failed. No retries permitted until 2026-11-03 21:45:56.348099643 +0000 UTC m=+1.700426451 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-oauth-serving-cert") pod "console-86cf959768-tf4fd" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723") : object "openshift-console"/"oauth-serving-cert" not registered Nov 03 21:45:55.848470 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.848157 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9f5c183e-d58a-4621-940c-04a7ae0bafc9-hosts-file\") pod \"node-resolver-9pctw\" (UID: \"9f5c183e-d58a-4621-940c-04a7ae0bafc9\") " pod="openshift-dns/node-resolver-9pctw" Nov 03 21:45:55.848470 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.848175 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-host-cni-bin\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.848470 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.848193 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/00c924c1-ac0d-49c8-8610-3b837fbb8277-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:55.848470 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.848211 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:55.848470 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.848231 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/00c924c1-ac0d-49c8-8610-3b837fbb8277-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:55.848470 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.848234 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:55.848470 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.848249 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/32fd1d86-45ed-4a7d-894e-3990980bc237-cert\") pod \"ingress-canary-77wdg\" (UID: \"32fd1d86-45ed-4a7d-894e-3990980bc237\") " pod="openshift-ingress-canary/ingress-canary-77wdg" Nov 03 21:45:55.848470 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.848306 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e652c3ec-d5ac-43f9-ade5-d3da32041cee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-285hg\" (UID: \"e652c3ec-d5ac-43f9-ade5-d3da32041cee\") " pod="openshift-insights/insights-runtime-extractor-285hg" Nov 03 21:45:55.848470 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.848410 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:55.848470 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.848438 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:55.849295 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.848510 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfkkl\" (UniqueName: \"kubernetes.io/projected/a6cf0029-66be-4d86-ae8e-713d173a113e-kube-api-access-zfkkl\") pod \"monitoring-plugin-77f4f76594-9cnqh\" (UID: \"a6cf0029-66be-4d86-ae8e-713d173a113e\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-9cnqh" Nov 03 21:45:55.849295 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.848527 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/00c924c1-ac0d-49c8-8610-3b837fbb8277-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:55.849295 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.848542 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:55.849295 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.848564 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-run-ovn\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.849295 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.848587 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-trusted-ca-bundle\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:55.849295 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.848608 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-etc-kubernetes\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:45:55.849295 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.848623 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-etc-systemd\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:45:55.849295 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.848638 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-var-lib-kubelet\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:45:55.849295 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.848648 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/76122905-15b4-4f4e-97b9-9f73a7675e36-ovn-node-metrics-cert\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.849295 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.848675 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:55.849295 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.848705 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/00c924c1-ac0d-49c8-8610-3b837fbb8277-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:55.849295 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:55.848713 6524 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered Nov 03 21:45:55.849295 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.848701 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9f5c183e-d58a-4621-940c-04a7ae0bafc9-hosts-file\") pod \"node-resolver-9pctw\" (UID: \"9f5c183e-d58a-4621-940c-04a7ae0bafc9\") " pod="openshift-dns/node-resolver-9pctw" Nov 03 21:45:55.849295 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.848829 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:55.849295 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:55.848853 6524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-trusted-ca-bundle podName:866ddb5a-c929-40c1-911d-ca40c5405723 nodeName:}" failed. No retries permitted until 2026-11-03 21:45:56.348838407 +0000 UTC m=+1.701165224 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-trusted-ca-bundle") pod "console-86cf959768-tf4fd" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723") : object "openshift-console"/"trusted-ca-bundle" not registered Nov 03 21:45:55.849295 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.848855 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:55.849295 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.848883 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:55.849295 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.848953 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/32fd1d86-45ed-4a7d-894e-3990980bc237-cert\") pod \"ingress-canary-77wdg\" (UID: \"32fd1d86-45ed-4a7d-894e-3990980bc237\") " pod="openshift-ingress-canary/ingress-canary-77wdg" Nov 03 21:45:55.849295 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.848991 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5kf4\" (UniqueName: \"kubernetes.io/projected/866ddb5a-c929-40c1-911d-ca40c5405723-kube-api-access-n5kf4\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:55.849295 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.849036 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-etc-modprobe-d\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:45:55.849295 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.849052 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:55.849295 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.849140 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0db6d946-a1d8-4459-a883-6f93f7e1182a-iptables-alerter-script\") pod \"iptables-alerter-nkngm\" (UID: \"0db6d946-a1d8-4459-a883-6f93f7e1182a\") " pod="openshift-network-operator/iptables-alerter-nkngm" Nov 03 21:45:55.849295 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.849164 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/48d8b783-b602-47ef-9b9b-f2416d448908-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wvd2c\" (UID: \"48d8b783-b602-47ef-9b9b-f2416d448908\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" Nov 03 21:45:55.849295 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.849238 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7xhd\" (UniqueName: \"kubernetes.io/projected/e652c3ec-d5ac-43f9-ade5-d3da32041cee-kube-api-access-p7xhd\") pod \"insights-runtime-extractor-285hg\" (UID: \"e652c3ec-d5ac-43f9-ade5-d3da32041cee\") " pod="openshift-insights/insights-runtime-extractor-285hg" Nov 03 21:45:55.849295 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.849286 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-node-exporter-wtmp\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:55.850391 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.849351 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0db6d946-a1d8-4459-a883-6f93f7e1182a-iptables-alerter-script\") pod \"iptables-alerter-nkngm\" (UID: \"0db6d946-a1d8-4459-a883-6f93f7e1182a\") " pod="openshift-network-operator/iptables-alerter-nkngm" Nov 03 21:45:55.850391 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.849383 6524 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 03 21:45:55.850391 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.849423 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:55.850391 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.849506 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/94aaa3ac-c7fc-4f87-878e-8ef79127488b-dbus\") pod \"global-pull-secret-syncer-jq6hj\" (UID: \"94aaa3ac-c7fc-4f87-878e-8ef79127488b\") " pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:45:55.850391 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:55.849569 6524 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-137-7.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-7.us-east-2.compute.internal" Nov 03 21:45:55.850391 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.849642 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e652c3ec-d5ac-43f9-ade5-d3da32041cee-data-volume\") pod \"insights-runtime-extractor-285hg\" (UID: \"e652c3ec-d5ac-43f9-ade5-d3da32041cee\") " pod="openshift-insights/insights-runtime-extractor-285hg" Nov 03 21:45:55.850391 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.849686 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:55.850391 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.849704 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e652c3ec-d5ac-43f9-ade5-d3da32041cee-data-volume\") pod \"insights-runtime-extractor-285hg\" (UID: \"e652c3ec-d5ac-43f9-ade5-d3da32041cee\") " pod="openshift-insights/insights-runtime-extractor-285hg" Nov 03 21:45:55.850391 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.849715 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-web-config\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:55.850391 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.849744 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/00c924c1-ac0d-49c8-8610-3b837fbb8277-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:55.850391 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.849771 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phmrg\" (UniqueName: \"kubernetes.io/projected/00c924c1-ac0d-49c8-8610-3b837fbb8277-kube-api-access-phmrg\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:55.850391 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.849794 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:55.850391 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.849817 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-node-log\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.850391 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.849838 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00c924c1-ac0d-49c8-8610-3b837fbb8277-metrics-client-ca\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:55.850391 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.849864 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/00c924c1-ac0d-49c8-8610-3b837fbb8277-secret-grpc-tls\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:55.850391 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.849888 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0db6d946-a1d8-4459-a883-6f93f7e1182a-host-slash\") pod \"iptables-alerter-nkngm\" (UID: \"0db6d946-a1d8-4459-a883-6f93f7e1182a\") " pod="openshift-network-operator/iptables-alerter-nkngm" Nov 03 21:45:55.850391 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.849894 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-web-config\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:55.850391 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.849913 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb5f720e-f465-439f-ba32-4f595561f9f6-metrics-certs\") pod \"network-metrics-daemon-knt57\" (UID: \"eb5f720e-f465-439f-ba32-4f595561f9f6\") " pod="openshift-multus/network-metrics-daemon-knt57" Nov 03 21:45:55.850391 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.849937 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-node-exporter-tls\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:55.850391 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.849941 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/00c924c1-ac0d-49c8-8610-3b837fbb8277-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:55.850391 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.850017 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:55.850391 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.850049 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-run-systemd\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.850391 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.850066 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-etc-sysctl-conf\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:45:55.850391 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.850152 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/00c924c1-ac0d-49c8-8610-3b837fbb8277-secret-grpc-tls\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:55.850391 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.850172 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-run\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:45:55.850391 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.850196 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb5f720e-f465-439f-ba32-4f595561f9f6-metrics-certs\") pod \"network-metrics-daemon-knt57\" (UID: \"eb5f720e-f465-439f-ba32-4f595561f9f6\") " pod="openshift-multus/network-metrics-daemon-knt57" Nov 03 21:45:55.850391 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.850204 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmr4b\" (UniqueName: \"kubernetes.io/projected/76122905-15b4-4f4e-97b9-9f73a7675e36-kube-api-access-tmr4b\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.850391 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.850197 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00c924c1-ac0d-49c8-8610-3b837fbb8277-metrics-client-ca\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:55.850391 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.850324 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/00c924c1-ac0d-49c8-8610-3b837fbb8277-secret-thanos-querier-tls\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:55.850391 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.850361 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-etc-sysconfig\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:45:55.851743 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.850420 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6c6w8\" (UniqueName: \"kubernetes.io/projected/0db6d946-a1d8-4459-a883-6f93f7e1182a-kube-api-access-6c6w8\") pod \"iptables-alerter-nkngm\" (UID: \"0db6d946-a1d8-4459-a883-6f93f7e1182a\") " pod="openshift-network-operator/iptables-alerter-nkngm" Nov 03 21:45:55.851743 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.850456 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-config-out\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:55.851743 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.850489 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:55.851743 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.850506 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00c924c1-ac0d-49c8-8610-3b837fbb8277-tmp\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:55.851743 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.850521 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:55.851743 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.850538 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:55.851743 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.850537 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/00c924c1-ac0d-49c8-8610-3b837fbb8277-secret-thanos-querier-tls\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:55.851743 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.850561 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-config-out\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:55.851743 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.850354 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-node-exporter-tls\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:55.851743 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.850611 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00c924c1-ac0d-49c8-8610-3b837fbb8277-tmp\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:55.851743 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.850638 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:55.851743 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.850674 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a440580e-b0bd-49e2-a33d-576118e049ae-serviceca\") pod \"node-ca-nz4zw\" (UID: \"a440580e-b0bd-49e2-a33d-576118e049ae\") " pod="openshift-image-registry/node-ca-nz4zw" Nov 03 21:45:55.851743 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.850715 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-host-var-lib-kubelet\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.851743 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.850758 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:55.851743 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.850766 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/48d8b783-b602-47ef-9b9b-f2416d448908-sys-fs\") pod \"aws-ebs-csi-driver-node-wvd2c\" (UID: \"48d8b783-b602-47ef-9b9b-f2416d448908\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" Nov 03 21:45:55.851743 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.850777 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:55.851743 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.850795 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e652c3ec-d5ac-43f9-ade5-d3da32041cee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-285hg\" (UID: \"e652c3ec-d5ac-43f9-ade5-d3da32041cee\") " pod="openshift-insights/insights-runtime-extractor-285hg" Nov 03 21:45:55.851743 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.850797 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:55.851743 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.850836 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-node-exporter-accelerators-collector-config\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:55.851743 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.850862 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:55.851743 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.850912 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:55.851743 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.851040 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlgsq\" (UniqueName: \"kubernetes.io/projected/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-kube-api-access-mlgsq\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:55.851743 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.851061 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-node-exporter-accelerators-collector-config\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:55.851743 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.851072 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e652c3ec-d5ac-43f9-ade5-d3da32041cee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-285hg\" (UID: \"e652c3ec-d5ac-43f9-ade5-d3da32041cee\") " pod="openshift-insights/insights-runtime-extractor-285hg" Nov 03 21:45:55.851743 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.851112 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/76122905-15b4-4f4e-97b9-9f73a7675e36-ovnkube-script-lib\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.851743 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.851305 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:55.851743 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.851418 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a440580e-b0bd-49e2-a33d-576118e049ae-serviceca\") pod \"node-ca-nz4zw\" (UID: \"a440580e-b0bd-49e2-a33d-576118e049ae\") " pod="openshift-image-registry/node-ca-nz4zw" Nov 03 21:45:55.851743 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.851428 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:55.851743 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.851476 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:55.851743 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.851549 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7g58\" (UniqueName: \"kubernetes.io/projected/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-kube-api-access-s7g58\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:55.851743 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.851626 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/76122905-15b4-4f4e-97b9-9f73a7675e36-ovnkube-script-lib\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.851743 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.851666 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-host-run-netns\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.851743 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.851671 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:55.851743 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.851720 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.851768 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/219625cd-f1a4-4e51-90ee-58f0fc5939bb-cni-binary-copy\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.851826 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8fqs\" (UniqueName: \"kubernetes.io/projected/48d8b783-b602-47ef-9b9b-f2416d448908-kube-api-access-k8fqs\") pod \"aws-ebs-csi-driver-node-wvd2c\" (UID: \"48d8b783-b602-47ef-9b9b-f2416d448908\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.851864 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-multus-socket-dir-parent\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.851895 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-host-run-netns\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.851948 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/48d8b783-b602-47ef-9b9b-f2416d448908-device-dir\") pod \"aws-ebs-csi-driver-node-wvd2c\" (UID: \"48d8b783-b602-47ef-9b9b-f2416d448908\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.851950 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/219625cd-f1a4-4e51-90ee-58f0fc5939bb-cni-binary-copy\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.851988 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-host-kubelet\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.852021 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-run-openvswitch\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.852050 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-host-run-ovn-kubernetes\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.852073 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-host-cni-netd\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.852150 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/76122905-15b4-4f4e-97b9-9f73a7675e36-ovnkube-config\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.852188 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-multus-conf-dir\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.852225 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.852252 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-serving-cert\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.852313 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/76122905-15b4-4f4e-97b9-9f73a7675e36-ovnkube-config\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.852335 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-console-config\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:55.852317 6524 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.852364 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-host-var-lib-cni-multus\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.852390 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-host-run-multus-certs\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:55.852424 6524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-serving-cert podName:866ddb5a-c929-40c1-911d-ca40c5405723 nodeName:}" failed. No retries permitted until 2026-11-03 21:45:56.35240599 +0000 UTC m=+1.704732793 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-serving-cert") pod "console-86cf959768-tf4fd" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723") : object "openshift-console"/"console-serving-cert" not registered Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:55.852428 6524 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.852446 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgzcx\" (UniqueName: \"kubernetes.io/projected/eb5f720e-f465-439f-ba32-4f595561f9f6-kube-api-access-pgzcx\") pod \"network-metrics-daemon-knt57\" (UID: \"eb5f720e-f465-439f-ba32-4f595561f9f6\") " pod="openshift-multus/network-metrics-daemon-knt57" Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.852475 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4cz4g\" (UID: \"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c\") " pod="openshift-multus/multus-additional-cni-plugins-4cz4g" Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:55.852495 6524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-console-config podName:866ddb5a-c929-40c1-911d-ca40c5405723 nodeName:}" failed. No retries permitted until 2026-11-03 21:45:56.352481685 +0000 UTC m=+1.704808473 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-console-config") pod "console-86cf959768-tf4fd" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723") : object "openshift-console"/"console-config" not registered Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.852525 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/811514f5-baff-4fc1-9e92-28f27de7b0b7-agent-certs\") pod \"konnectivity-agent-v98zn\" (UID: \"811514f5-baff-4fc1-9e92-28f27de7b0b7\") " pod="kube-system/konnectivity-agent-v98zn" Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.852543 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/94aaa3ac-c7fc-4f87-878e-8ef79127488b-kubelet-config\") pod \"global-pull-secret-syncer-jq6hj\" (UID: \"94aaa3ac-c7fc-4f87-878e-8ef79127488b\") " pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.852578 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-var-lib-openvswitch\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.852661 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-oauth-config\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.852680 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-etc-sysctl-d\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.852700 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-sys\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.852719 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9c970a61-3ebf-48c9-a05b-688accd0b18a-tmp\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.852736 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-config\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.852752 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-host-var-lib-cni-bin\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.852772 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c-cnibin\") pod \"multus-additional-cni-plugins-4cz4g\" (UID: \"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c\") " pod="openshift-multus/multus-additional-cni-plugins-4cz4g" Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:55.852735 6524 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.852802 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csdhx\" (UniqueName: \"kubernetes.io/projected/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c-kube-api-access-csdhx\") pod \"multus-additional-cni-plugins-4cz4g\" (UID: \"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c\") " pod="openshift-multus/multus-additional-cni-plugins-4cz4g" Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.852819 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/811514f5-baff-4fc1-9e92-28f27de7b0b7-agent-certs\") pod \"konnectivity-agent-v98zn\" (UID: \"811514f5-baff-4fc1-9e92-28f27de7b0b7\") " pod="kube-system/konnectivity-agent-v98zn" Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.852827 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/48d8b783-b602-47ef-9b9b-f2416d448908-socket-dir\") pod \"aws-ebs-csi-driver-node-wvd2c\" (UID: \"48d8b783-b602-47ef-9b9b-f2416d448908\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:55.852838 6524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-oauth-config podName:866ddb5a-c929-40c1-911d-ca40c5405723 nodeName:}" failed. No retries permitted until 2026-11-03 21:45:56.352821913 +0000 UTC m=+1.705148697 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-oauth-config") pod "console-86cf959768-tf4fd" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723") : object "openshift-console"/"console-oauth-config" not registered Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.852838 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9c970a61-3ebf-48c9-a05b-688accd0b18a-tmp\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.852896 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.852955 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-multus-cni-dir\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.853034 6524 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.853215 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-config\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.853229 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.853286 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-metrics-client-ca\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.853325 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-etc-openvswitch\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.853362 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.853348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.853388 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-host\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.853413 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5921d9cf-54a7-43bd-b697-c5ccf9f39135-config-volume\") pod \"dns-default-x2r5s\" (UID: \"5921d9cf-54a7-43bd-b697-c5ccf9f39135\") " pod="openshift-dns/dns-default-x2r5s" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.853438 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5921d9cf-54a7-43bd-b697-c5ccf9f39135-metrics-tls\") pod \"dns-default-x2r5s\" (UID: \"5921d9cf-54a7-43bd-b697-c5ccf9f39135\") " pod="openshift-dns/dns-default-x2r5s" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.853485 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/00c924c1-ac0d-49c8-8610-3b837fbb8277-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.853498 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.853510 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.853552 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e652c3ec-d5ac-43f9-ade5-d3da32041cee-crio-socket\") pod \"insights-runtime-extractor-285hg\" (UID: \"e652c3ec-d5ac-43f9-ade5-d3da32041cee\") " pod="openshift-insights/insights-runtime-extractor-285hg" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.853585 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/219625cd-f1a4-4e51-90ee-58f0fc5939bb-multus-daemon-config\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.853654 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl5nr\" (UniqueName: \"kubernetes.io/projected/219625cd-f1a4-4e51-90ee-58f0fc5939bb-kube-api-access-kl5nr\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.853686 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-config-volume\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.853714 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.853740 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9c970a61-3ebf-48c9-a05b-688accd0b18a-etc-tuned\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.853744 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.853769 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-log-socket\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.853714 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.853820 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/219625cd-f1a4-4e51-90ee-58f0fc5939bb-multus-daemon-config\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:55.853830 6524 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-7.us-east-2.compute.internal" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.853863 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-lib-modules\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.853909 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-metrics-client-ca\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.853741 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.853953 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5921d9cf-54a7-43bd-b697-c5ccf9f39135-config-volume\") pod \"dns-default-x2r5s\" (UID: \"5921d9cf-54a7-43bd-b697-c5ccf9f39135\") " pod="openshift-dns/dns-default-x2r5s" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.853997 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-config-volume\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.854010 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9c970a61-3ebf-48c9-a05b-688accd0b18a-etc-tuned\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.854017 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a440580e-b0bd-49e2-a33d-576118e049ae-host\") pod \"node-ca-nz4zw\" (UID: \"a440580e-b0bd-49e2-a33d-576118e049ae\") " pod="openshift-image-registry/node-ca-nz4zw" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.854044 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-host-run-k8s-cni-cncf-io\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.854096 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.854162 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/00c924c1-ac0d-49c8-8610-3b837fbb8277-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.854175 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/811514f5-baff-4fc1-9e92-28f27de7b0b7-konnectivity-ca\") pod \"konnectivity-agent-v98zn\" (UID: \"811514f5-baff-4fc1-9e92-28f27de7b0b7\") " pod="kube-system/konnectivity-agent-v98zn" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.854226 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-node-exporter-textfile\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.854281 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5921d9cf-54a7-43bd-b697-c5ccf9f39135-metrics-tls\") pod \"dns-default-x2r5s\" (UID: \"5921d9cf-54a7-43bd-b697-c5ccf9f39135\") " pod="openshift-dns/dns-default-x2r5s" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.854302 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.854337 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-node-exporter-textfile\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.854385 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9k8xx\" (UniqueName: \"kubernetes.io/projected/9c970a61-3ebf-48c9-a05b-688accd0b18a-kube-api-access-9k8xx\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.854412 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/811514f5-baff-4fc1-9e92-28f27de7b0b7-konnectivity-ca\") pod \"konnectivity-agent-v98zn\" (UID: \"811514f5-baff-4fc1-9e92-28f27de7b0b7\") " pod="kube-system/konnectivity-agent-v98zn" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.854413 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.854544 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4cz4g\" (UID: \"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c\") " pod="openshift-multus/multus-additional-cni-plugins-4cz4g" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.854627 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/48d8b783-b602-47ef-9b9b-f2416d448908-etc-selinux\") pod \"aws-ebs-csi-driver-node-wvd2c\" (UID: \"48d8b783-b602-47ef-9b9b-f2416d448908\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.854668 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-sys\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.854706 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-config-out\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.854757 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.854782 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4cz4g\" (UID: \"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c\") " pod="openshift-multus/multus-additional-cni-plugins-4cz4g" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.854791 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.854784 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-config-out\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.854825 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/94aaa3ac-c7fc-4f87-878e-8ef79127488b-original-pull-secret\") pod \"global-pull-secret-syncer-jq6hj\" (UID: \"94aaa3ac-c7fc-4f87-878e-8ef79127488b\") " pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.854857 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-host-slash\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.854884 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-service-ca\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.854914 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-system-cni-dir\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.854929 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.854936 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jl4cx\" (UniqueName: \"kubernetes.io/projected/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-kube-api-access-jl4cx\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:55.854939 6524 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.854959 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a6cf0029-66be-4d86-ae8e-713d173a113e-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-9cnqh\" (UID: \"a6cf0029-66be-4d86-ae8e-713d173a113e\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-9cnqh" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:55.854986 6524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-service-ca podName:866ddb5a-c929-40c1-911d-ca40c5405723 nodeName:}" failed. No retries permitted until 2026-11-03 21:45:56.354970915 +0000 UTC m=+1.707297711 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-service-ca") pod "console-86cf959768-tf4fd" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723") : object "openshift-console"/"service-ca" not registered Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.854999 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.855003 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.855055 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.855175 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a6cf0029-66be-4d86-ae8e-713d173a113e-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-9cnqh\" (UID: \"a6cf0029-66be-4d86-ae8e-713d173a113e\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-9cnqh" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.855193 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.855180 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/94aaa3ac-c7fc-4f87-878e-8ef79127488b-original-pull-secret\") pod \"global-pull-secret-syncer-jq6hj\" (UID: \"94aaa3ac-c7fc-4f87-878e-8ef79127488b\") " pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:45:55.855429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.855298 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:55.858538 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.858521 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqplq\" (UniqueName: \"kubernetes.io/projected/a440580e-b0bd-49e2-a33d-576118e049ae-kube-api-access-lqplq\") pod \"node-ca-nz4zw\" (UID: \"a440580e-b0bd-49e2-a33d-576118e049ae\") " pod="openshift-image-registry/node-ca-nz4zw" Nov 03 21:45:55.859381 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.859362 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8h9g\" (UniqueName: \"kubernetes.io/projected/32fd1d86-45ed-4a7d-894e-3990980bc237-kube-api-access-m8h9g\") pod \"ingress-canary-77wdg\" (UID: \"32fd1d86-45ed-4a7d-894e-3990980bc237\") " pod="openshift-ingress-canary/ingress-canary-77wdg" Nov 03 21:45:55.860503 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:55.860482 6524 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered Nov 03 21:45:55.860624 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:55.860505 6524 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered Nov 03 21:45:55.860624 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:55.860521 6524 projected.go:196] Error preparing data for projected volume kube-api-access-n5kf4 for pod openshift-console/console-86cf959768-tf4fd: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Nov 03 21:45:55.860624 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:55.860582 6524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/866ddb5a-c929-40c1-911d-ca40c5405723-kube-api-access-n5kf4 podName:866ddb5a-c929-40c1-911d-ca40c5405723 nodeName:}" failed. No retries permitted until 2026-11-03 21:45:56.360564751 +0000 UTC m=+1.712891555 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-n5kf4" (UniqueName: "kubernetes.io/projected/866ddb5a-c929-40c1-911d-ca40c5405723-kube-api-access-n5kf4") pod "console-86cf959768-tf4fd" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Nov 03 21:45:55.860624 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.860579 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:55.860929 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.860908 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7xhd\" (UniqueName: \"kubernetes.io/projected/e652c3ec-d5ac-43f9-ade5-d3da32041cee-kube-api-access-p7xhd\") pod \"insights-runtime-extractor-285hg\" (UID: \"e652c3ec-d5ac-43f9-ade5-d3da32041cee\") " pod="openshift-insights/insights-runtime-extractor-285hg" Nov 03 21:45:55.860985 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.860968 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfkkl\" (UniqueName: \"kubernetes.io/projected/a6cf0029-66be-4d86-ae8e-713d173a113e-kube-api-access-zfkkl\") pod \"monitoring-plugin-77f4f76594-9cnqh\" (UID: \"a6cf0029-66be-4d86-ae8e-713d173a113e\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-9cnqh" Nov 03 21:45:55.861095 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.861072 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmr4b\" (UniqueName: \"kubernetes.io/projected/76122905-15b4-4f4e-97b9-9f73a7675e36-kube-api-access-tmr4b\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.861202 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.861162 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6c6w8\" (UniqueName: \"kubernetes.io/projected/0db6d946-a1d8-4459-a883-6f93f7e1182a-kube-api-access-6c6w8\") pod \"iptables-alerter-nkngm\" (UID: \"0db6d946-a1d8-4459-a883-6f93f7e1182a\") " pod="openshift-network-operator/iptables-alerter-nkngm" Nov 03 21:45:55.861418 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.861393 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-phmrg\" (UniqueName: \"kubernetes.io/projected/00c924c1-ac0d-49c8-8610-3b837fbb8277-kube-api-access-phmrg\") pod \"thanos-querier-54674f497-2g5ct\" (UID: \"00c924c1-ac0d-49c8-8610-3b837fbb8277\") " pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:55.861614 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.861596 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlgsq\" (UniqueName: \"kubernetes.io/projected/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-kube-api-access-mlgsq\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:55.861652 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.861614 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:55.862272 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.862235 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7g58\" (UniqueName: \"kubernetes.io/projected/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-kube-api-access-s7g58\") pod \"alertmanager-main-0\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:45:55.862479 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.862458 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7m8x\" (UniqueName: \"kubernetes.io/projected/5921d9cf-54a7-43bd-b697-c5ccf9f39135-kube-api-access-t7m8x\") pod \"dns-default-x2r5s\" (UID: \"5921d9cf-54a7-43bd-b697-c5ccf9f39135\") " pod="openshift-dns/dns-default-x2r5s" Nov 03 21:45:55.862642 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.862625 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgzcx\" (UniqueName: \"kubernetes.io/projected/eb5f720e-f465-439f-ba32-4f595561f9f6-kube-api-access-pgzcx\") pod \"network-metrics-daemon-knt57\" (UID: \"eb5f720e-f465-439f-ba32-4f595561f9f6\") " pod="openshift-multus/network-metrics-daemon-knt57" Nov 03 21:45:55.862747 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.862732 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:55.863317 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.863295 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-csdhx\" (UniqueName: \"kubernetes.io/projected/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c-kube-api-access-csdhx\") pod \"multus-additional-cni-plugins-4cz4g\" (UID: \"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c\") " pod="openshift-multus/multus-additional-cni-plugins-4cz4g" Nov 03 21:45:55.863733 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.863713 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8fqs\" (UniqueName: \"kubernetes.io/projected/48d8b783-b602-47ef-9b9b-f2416d448908-kube-api-access-k8fqs\") pod \"aws-ebs-csi-driver-node-wvd2c\" (UID: \"48d8b783-b602-47ef-9b9b-f2416d448908\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" Nov 03 21:45:55.867965 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.867942 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jl4cx\" (UniqueName: \"kubernetes.io/projected/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-kube-api-access-jl4cx\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:55.868041 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.867972 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9k8xx\" (UniqueName: \"kubernetes.io/projected/9c970a61-3ebf-48c9-a05b-688accd0b18a-kube-api-access-9k8xx\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:45:55.868109 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.868090 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl5nr\" (UniqueName: \"kubernetes.io/projected/219625cd-f1a4-4e51-90ee-58f0fc5939bb-kube-api-access-kl5nr\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.956176 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956114 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-sys\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:55.956176 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956173 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-host-slash\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.956366 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956198 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-system-cni-dir\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.956366 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956233 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-sys\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:55.956366 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956259 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-systemd-units\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.956366 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956300 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c-system-cni-dir\") pod \"multus-additional-cni-plugins-4cz4g\" (UID: \"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c\") " pod="openshift-multus/multus-additional-cni-plugins-4cz4g" Nov 03 21:45:55.956366 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956306 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-systemd-units\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.956366 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956251 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-system-cni-dir\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.956366 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956239 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-host-slash\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.956366 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956329 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/48d8b783-b602-47ef-9b9b-f2416d448908-registration-dir\") pod \"aws-ebs-csi-driver-node-wvd2c\" (UID: \"48d8b783-b602-47ef-9b9b-f2416d448908\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" Nov 03 21:45:55.956366 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956360 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c-system-cni-dir\") pod \"multus-additional-cni-plugins-4cz4g\" (UID: \"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c\") " pod="openshift-multus/multus-additional-cni-plugins-4cz4g" Nov 03 21:45:55.956716 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956374 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-etc-kubernetes\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.956716 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956387 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/48d8b783-b602-47ef-9b9b-f2416d448908-registration-dir\") pod \"aws-ebs-csi-driver-node-wvd2c\" (UID: \"48d8b783-b602-47ef-9b9b-f2416d448908\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" Nov 03 21:45:55.956716 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956417 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-cnibin\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.956716 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956432 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-hostroot\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.956716 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956453 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-root\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:55.956716 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956457 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-etc-kubernetes\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.956716 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956482 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-hostroot\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.956716 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956491 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-host-cni-bin\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.956716 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956496 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-root\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:55.956716 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956513 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-run-ovn\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.956716 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956510 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-cnibin\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.956716 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956526 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-host-cni-bin\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.956716 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956537 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-etc-kubernetes\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:45:55.956716 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956553 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-etc-systemd\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:45:55.956716 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956572 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-var-lib-kubelet\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:45:55.956716 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956592 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-run-ovn\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.956716 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956609 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-var-lib-kubelet\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:45:55.956716 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956621 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-etc-systemd\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:45:55.956716 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956628 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-etc-modprobe-d\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:45:55.956716 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956643 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-etc-kubernetes\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:45:55.956716 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956657 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/48d8b783-b602-47ef-9b9b-f2416d448908-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wvd2c\" (UID: \"48d8b783-b602-47ef-9b9b-f2416d448908\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" Nov 03 21:45:55.956716 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956697 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/48d8b783-b602-47ef-9b9b-f2416d448908-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wvd2c\" (UID: \"48d8b783-b602-47ef-9b9b-f2416d448908\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" Nov 03 21:45:55.956716 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956701 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-node-exporter-wtmp\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:55.956716 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956731 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-etc-modprobe-d\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956745 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/94aaa3ac-c7fc-4f87-878e-8ef79127488b-dbus\") pod \"global-pull-secret-syncer-jq6hj\" (UID: \"94aaa3ac-c7fc-4f87-878e-8ef79127488b\") " pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956770 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-node-log\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956786 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0db6d946-a1d8-4459-a883-6f93f7e1182a-host-slash\") pod \"iptables-alerter-nkngm\" (UID: \"0db6d946-a1d8-4459-a883-6f93f7e1182a\") " pod="openshift-network-operator/iptables-alerter-nkngm" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956800 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bfd8df86-d3d0-4576-afd0-fa742a6a3aad-node-exporter-wtmp\") pod \"node-exporter-jdbs4\" (UID: \"bfd8df86-d3d0-4576-afd0-fa742a6a3aad\") " pod="openshift-monitoring/node-exporter-jdbs4" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956809 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-run-systemd\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956833 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0db6d946-a1d8-4459-a883-6f93f7e1182a-host-slash\") pod \"iptables-alerter-nkngm\" (UID: \"0db6d946-a1d8-4459-a883-6f93f7e1182a\") " pod="openshift-network-operator/iptables-alerter-nkngm" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956833 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-etc-sysctl-conf\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956838 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-node-log\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956853 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-run\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956855 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/94aaa3ac-c7fc-4f87-878e-8ef79127488b-dbus\") pod \"global-pull-secret-syncer-jq6hj\" (UID: \"94aaa3ac-c7fc-4f87-878e-8ef79127488b\") " pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956870 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-etc-sysconfig\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956877 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-run-systemd\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956907 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-etc-sysconfig\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956912 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-host-var-lib-kubelet\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956973 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-etc-sysctl-conf\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.956976 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-run\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957007 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/48d8b783-b602-47ef-9b9b-f2416d448908-sys-fs\") pod \"aws-ebs-csi-driver-node-wvd2c\" (UID: \"48d8b783-b602-47ef-9b9b-f2416d448908\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957030 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-host-run-netns\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957046 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-multus-socket-dir-parent\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957060 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-host-run-netns\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957076 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/48d8b783-b602-47ef-9b9b-f2416d448908-device-dir\") pod \"aws-ebs-csi-driver-node-wvd2c\" (UID: \"48d8b783-b602-47ef-9b9b-f2416d448908\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957088 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-host-run-netns\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957027 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-host-var-lib-kubelet\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957059 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/48d8b783-b602-47ef-9b9b-f2416d448908-sys-fs\") pod \"aws-ebs-csi-driver-node-wvd2c\" (UID: \"48d8b783-b602-47ef-9b9b-f2416d448908\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957107 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-host-run-netns\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957091 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-host-kubelet\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957150 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-host-kubelet\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957168 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-multus-socket-dir-parent\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957176 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-run-openvswitch\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957197 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-host-run-ovn-kubernetes\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957176 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/48d8b783-b602-47ef-9b9b-f2416d448908-device-dir\") pod \"aws-ebs-csi-driver-node-wvd2c\" (UID: \"48d8b783-b602-47ef-9b9b-f2416d448908\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957223 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-run-openvswitch\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957236 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-host-cni-netd\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957250 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-host-run-ovn-kubernetes\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957257 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-multus-conf-dir\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957263 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-host-cni-netd\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957288 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-host-var-lib-cni-multus\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957303 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-host-run-multus-certs\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957321 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4cz4g\" (UID: \"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c\") " pod="openshift-multus/multus-additional-cni-plugins-4cz4g" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957336 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/94aaa3ac-c7fc-4f87-878e-8ef79127488b-kubelet-config\") pod \"global-pull-secret-syncer-jq6hj\" (UID: \"94aaa3ac-c7fc-4f87-878e-8ef79127488b\") " pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957337 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-host-var-lib-cni-multus\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957355 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-multus-conf-dir\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957364 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-var-lib-openvswitch\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957390 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/94aaa3ac-c7fc-4f87-878e-8ef79127488b-kubelet-config\") pod \"global-pull-secret-syncer-jq6hj\" (UID: \"94aaa3ac-c7fc-4f87-878e-8ef79127488b\") " pod="kube-system/global-pull-secret-syncer-jq6hj" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957427 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4cz4g\" (UID: \"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c\") " pod="openshift-multus/multus-additional-cni-plugins-4cz4g" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957393 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-etc-sysctl-d\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957445 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-host-run-multus-certs\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957450 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-var-lib-openvswitch\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957465 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-sys\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957483 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-host-var-lib-cni-bin\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957500 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c-cnibin\") pod \"multus-additional-cni-plugins-4cz4g\" (UID: \"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c\") " pod="openshift-multus/multus-additional-cni-plugins-4cz4g" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957499 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-sys\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957513 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-host-var-lib-cni-bin\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957482 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-etc-sysctl-d\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957525 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/48d8b783-b602-47ef-9b9b-f2416d448908-socket-dir\") pod \"aws-ebs-csi-driver-node-wvd2c\" (UID: \"48d8b783-b602-47ef-9b9b-f2416d448908\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957526 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c-cnibin\") pod \"multus-additional-cni-plugins-4cz4g\" (UID: \"62e082c7-e3d4-4b77-a2ea-b8f41fc9e41c\") " pod="openshift-multus/multus-additional-cni-plugins-4cz4g" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957551 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-multus-cni-dir\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957565 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/48d8b783-b602-47ef-9b9b-f2416d448908-socket-dir\") pod \"aws-ebs-csi-driver-node-wvd2c\" (UID: \"48d8b783-b602-47ef-9b9b-f2416d448908\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957567 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-etc-openvswitch\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957591 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-etc-openvswitch\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.957571 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957592 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.959306 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957617 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-host\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:45:55.959306 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957624 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-multus-cni-dir\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.959306 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957618 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.959306 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957642 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-host\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:45:55.959306 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957643 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e652c3ec-d5ac-43f9-ade5-d3da32041cee-crio-socket\") pod \"insights-runtime-extractor-285hg\" (UID: \"e652c3ec-d5ac-43f9-ade5-d3da32041cee\") " pod="openshift-insights/insights-runtime-extractor-285hg" Nov 03 21:45:55.959306 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957665 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-log-socket\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.959306 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957668 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e652c3ec-d5ac-43f9-ade5-d3da32041cee-crio-socket\") pod \"insights-runtime-extractor-285hg\" (UID: \"e652c3ec-d5ac-43f9-ade5-d3da32041cee\") " pod="openshift-insights/insights-runtime-extractor-285hg" Nov 03 21:45:55.959306 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957680 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-lib-modules\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:45:55.959306 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957692 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/76122905-15b4-4f4e-97b9-9f73a7675e36-log-socket\") pod \"ovnkube-node-x26mj\" (UID: \"76122905-15b4-4f4e-97b9-9f73a7675e36\") " pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:55.959306 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957695 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a440580e-b0bd-49e2-a33d-576118e049ae-host\") pod \"node-ca-nz4zw\" (UID: \"a440580e-b0bd-49e2-a33d-576118e049ae\") " pod="openshift-image-registry/node-ca-nz4zw" Nov 03 21:45:55.959306 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957711 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-host-run-k8s-cni-cncf-io\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.959306 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957728 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/48d8b783-b602-47ef-9b9b-f2416d448908-etc-selinux\") pod \"aws-ebs-csi-driver-node-wvd2c\" (UID: \"48d8b783-b602-47ef-9b9b-f2416d448908\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" Nov 03 21:45:55.959306 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957736 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a440580e-b0bd-49e2-a33d-576118e049ae-host\") pod \"node-ca-nz4zw\" (UID: \"a440580e-b0bd-49e2-a33d-576118e049ae\") " pod="openshift-image-registry/node-ca-nz4zw" Nov 03 21:45:55.959306 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957776 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9c970a61-3ebf-48c9-a05b-688accd0b18a-lib-modules\") pod \"tuned-xw7xr\" (UID: \"9c970a61-3ebf-48c9-a05b-688accd0b18a\") " pod="openshift-cluster-node-tuning-operator/tuned-xw7xr" Nov 03 21:45:55.959306 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957778 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/219625cd-f1a4-4e51-90ee-58f0fc5939bb-host-run-k8s-cni-cncf-io\") pod \"multus-mbksx\" (UID: \"219625cd-f1a4-4e51-90ee-58f0fc5939bb\") " pod="openshift-multus/multus-mbksx" Nov 03 21:45:55.959306 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:55.957780 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/48d8b783-b602-47ef-9b9b-f2416d448908-etc-selinux\") pod \"aws-ebs-csi-driver-node-wvd2c\" (UID: \"48d8b783-b602-47ef-9b9b-f2416d448908\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wvd2c" Nov 03 21:45:56.253430 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:56.253402 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:56.360190 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:56.360152 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-service-ca\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:56.360333 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:56.360198 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-oauth-serving-cert\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:56.360333 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:56.360232 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-trusted-ca-bundle\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:56.360333 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:56.360256 6524 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered Nov 03 21:45:56.360333 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:56.360299 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-serving-cert\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:56.360333 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:56.360318 6524 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered Nov 03 21:45:56.360333 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:56.360331 6524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-service-ca podName:866ddb5a-c929-40c1-911d-ca40c5405723 nodeName:}" failed. No retries permitted until 2026-11-03 21:45:57.360310595 +0000 UTC m=+2.712637378 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-service-ca") pod "console-86cf959768-tf4fd" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723") : object "openshift-console"/"service-ca" not registered Nov 03 21:45:56.360627 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:56.360355 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-console-config\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:56.360627 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:56.360379 6524 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered Nov 03 21:45:56.360627 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:56.360398 6524 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered Nov 03 21:45:56.360627 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:56.360384 6524 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered Nov 03 21:45:56.360627 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:56.360385 6524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-oauth-serving-cert podName:866ddb5a-c929-40c1-911d-ca40c5405723 nodeName:}" failed. No retries permitted until 2026-11-03 21:45:57.360370503 +0000 UTC m=+2.712697301 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-oauth-serving-cert") pod "console-86cf959768-tf4fd" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723") : object "openshift-console"/"oauth-serving-cert" not registered Nov 03 21:45:56.360627 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:56.360463 6524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-console-config podName:866ddb5a-c929-40c1-911d-ca40c5405723 nodeName:}" failed. No retries permitted until 2026-11-03 21:45:57.360456349 +0000 UTC m=+2.712783131 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-console-config") pod "console-86cf959768-tf4fd" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723") : object "openshift-console"/"console-config" not registered Nov 03 21:45:56.360627 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:56.360474 6524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-trusted-ca-bundle podName:866ddb5a-c929-40c1-911d-ca40c5405723 nodeName:}" failed. No retries permitted until 2026-11-03 21:45:57.360469704 +0000 UTC m=+2.712796486 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-trusted-ca-bundle") pod "console-86cf959768-tf4fd" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723") : object "openshift-console"/"trusted-ca-bundle" not registered Nov 03 21:45:56.360627 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:56.360488 6524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-serving-cert podName:866ddb5a-c929-40c1-911d-ca40c5405723 nodeName:}" failed. No retries permitted until 2026-11-03 21:45:57.360480626 +0000 UTC m=+2.712807450 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-serving-cert") pod "console-86cf959768-tf4fd" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723") : object "openshift-console"/"console-serving-cert" not registered Nov 03 21:45:56.360627 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:56.360516 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-oauth-config\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:56.360627 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:56.360600 6524 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered Nov 03 21:45:56.360627 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:56.360627 6524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-oauth-config podName:866ddb5a-c929-40c1-911d-ca40c5405723 nodeName:}" failed. No retries permitted until 2026-11-03 21:45:57.360620494 +0000 UTC m=+2.712947275 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-oauth-config") pod "console-86cf959768-tf4fd" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723") : object "openshift-console"/"console-oauth-config" not registered Nov 03 21:45:56.461635 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:56.461598 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5kf4\" (UniqueName: \"kubernetes.io/projected/866ddb5a-c929-40c1-911d-ca40c5405723-kube-api-access-n5kf4\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:56.461808 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:56.461786 6524 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered Nov 03 21:45:56.461866 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:56.461811 6524 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered Nov 03 21:45:56.461866 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:56.461825 6524 projected.go:196] Error preparing data for projected volume kube-api-access-n5kf4 for pod openshift-console/console-86cf959768-tf4fd: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Nov 03 21:45:56.461954 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:56.461894 6524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/866ddb5a-c929-40c1-911d-ca40c5405723-kube-api-access-n5kf4 podName:866ddb5a-c929-40c1-911d-ca40c5405723 nodeName:}" failed. No retries permitted until 2026-11-03 21:45:57.461874716 +0000 UTC m=+2.814201513 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-n5kf4" (UniqueName: "kubernetes.io/projected/866ddb5a-c929-40c1-911d-ca40c5405723-kube-api-access-n5kf4") pod "console-86cf959768-tf4fd" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Nov 03 21:45:57.370856 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:57.370812 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-oauth-config\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:57.371325 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:57.370873 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-service-ca\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:57.371325 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:57.370900 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-oauth-serving-cert\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:57.371325 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:57.370943 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-trusted-ca-bundle\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:57.371325 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:57.370994 6524 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered Nov 03 21:45:57.371325 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:57.371015 6524 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered Nov 03 21:45:57.371325 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:57.371037 6524 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered Nov 03 21:45:57.371325 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:57.371010 6524 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered Nov 03 21:45:57.371325 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:57.370999 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-serving-cert\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:57.371325 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:57.371059 6524 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered Nov 03 21:45:57.371325 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:57.371070 6524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-oauth-serving-cert podName:866ddb5a-c929-40c1-911d-ca40c5405723 nodeName:}" failed. No retries permitted until 2026-11-03 21:45:59.371056794 +0000 UTC m=+4.723383577 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-oauth-serving-cert") pod "console-86cf959768-tf4fd" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723") : object "openshift-console"/"oauth-serving-cert" not registered Nov 03 21:45:57.371325 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:57.371157 6524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-oauth-config podName:866ddb5a-c929-40c1-911d-ca40c5405723 nodeName:}" failed. No retries permitted until 2026-11-03 21:45:59.371140999 +0000 UTC m=+4.723467799 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-oauth-config") pod "console-86cf959768-tf4fd" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723") : object "openshift-console"/"console-oauth-config" not registered Nov 03 21:45:57.371325 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:57.371176 6524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-trusted-ca-bundle podName:866ddb5a-c929-40c1-911d-ca40c5405723 nodeName:}" failed. No retries permitted until 2026-11-03 21:45:59.371169796 +0000 UTC m=+4.723496578 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-trusted-ca-bundle") pod "console-86cf959768-tf4fd" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723") : object "openshift-console"/"trusted-ca-bundle" not registered Nov 03 21:45:57.371325 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:57.371190 6524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-service-ca podName:866ddb5a-c929-40c1-911d-ca40c5405723 nodeName:}" failed. No retries permitted until 2026-11-03 21:45:59.371183788 +0000 UTC m=+4.723510570 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-service-ca") pod "console-86cf959768-tf4fd" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723") : object "openshift-console"/"service-ca" not registered Nov 03 21:45:57.371325 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:57.371202 6524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-serving-cert podName:866ddb5a-c929-40c1-911d-ca40c5405723 nodeName:}" failed. No retries permitted until 2026-11-03 21:45:59.371197858 +0000 UTC m=+4.723524641 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-serving-cert") pod "console-86cf959768-tf4fd" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723") : object "openshift-console"/"console-serving-cert" not registered Nov 03 21:45:57.371325 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:57.371219 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-console-config\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:57.371325 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:57.371276 6524 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered Nov 03 21:45:57.371325 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:57.371302 6524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-console-config podName:866ddb5a-c929-40c1-911d-ca40c5405723 nodeName:}" failed. No retries permitted until 2026-11-03 21:45:59.371296425 +0000 UTC m=+4.723623219 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-console-config") pod "console-86cf959768-tf4fd" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723") : object "openshift-console"/"console-config" not registered Nov 03 21:45:57.472187 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:57.472155 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5kf4\" (UniqueName: \"kubernetes.io/projected/866ddb5a-c929-40c1-911d-ca40c5405723-kube-api-access-n5kf4\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:57.472357 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:57.472324 6524 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered Nov 03 21:45:57.472357 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:57.472349 6524 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered Nov 03 21:45:57.472481 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:57.472363 6524 projected.go:196] Error preparing data for projected volume kube-api-access-n5kf4 for pod openshift-console/console-86cf959768-tf4fd: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Nov 03 21:45:57.472481 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:57.472431 6524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/866ddb5a-c929-40c1-911d-ca40c5405723-kube-api-access-n5kf4 podName:866ddb5a-c929-40c1-911d-ca40c5405723 nodeName:}" failed. No retries permitted until 2026-11-03 21:45:59.472411213 +0000 UTC m=+4.824738012 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-n5kf4" (UniqueName: "kubernetes.io/projected/866ddb5a-c929-40c1-911d-ca40c5405723-kube-api-access-n5kf4") pod "console-86cf959768-tf4fd" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Nov 03 21:45:57.575535 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:57.575503 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:45:57.839051 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:57.837360 6524 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Nov 03 21:45:57.968349 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:57.968317 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-x2r5s" Nov 03 21:45:58.300796 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:58.300729 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:45:59.392320 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:59.392277 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-service-ca\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:59.392796 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:59.392336 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-oauth-serving-cert\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:59.392796 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:59.392366 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-trusted-ca-bundle\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:59.392796 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:59.392389 6524 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered Nov 03 21:45:59.392796 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:59.392444 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-serving-cert\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:59.392796 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:59.392461 6524 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered Nov 03 21:45:59.392796 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:59.392505 6524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-service-ca podName:866ddb5a-c929-40c1-911d-ca40c5405723 nodeName:}" failed. No retries permitted until 2026-11-03 21:46:03.392462796 +0000 UTC m=+8.744789591 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-service-ca") pod "console-86cf959768-tf4fd" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723") : object "openshift-console"/"service-ca" not registered Nov 03 21:45:59.392796 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:59.392528 6524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-oauth-serving-cert podName:866ddb5a-c929-40c1-911d-ca40c5405723 nodeName:}" failed. No retries permitted until 2026-11-03 21:46:03.392515131 +0000 UTC m=+8.744841938 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-oauth-serving-cert") pod "console-86cf959768-tf4fd" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723") : object "openshift-console"/"oauth-serving-cert" not registered Nov 03 21:45:59.392796 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:59.392541 6524 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered Nov 03 21:45:59.392796 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:59.392551 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-console-config\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:59.392796 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:59.392599 6524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-serving-cert podName:866ddb5a-c929-40c1-911d-ca40c5405723 nodeName:}" failed. No retries permitted until 2026-11-03 21:46:03.392586475 +0000 UTC m=+8.744913261 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-serving-cert") pod "console-86cf959768-tf4fd" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723") : object "openshift-console"/"console-serving-cert" not registered Nov 03 21:45:59.392796 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:59.392624 6524 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered Nov 03 21:45:59.392796 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:59.392662 6524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-console-config podName:866ddb5a-c929-40c1-911d-ca40c5405723 nodeName:}" failed. No retries permitted until 2026-11-03 21:46:03.392650046 +0000 UTC m=+8.744976844 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-console-config") pod "console-86cf959768-tf4fd" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723") : object "openshift-console"/"console-config" not registered Nov 03 21:45:59.392796 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:59.392621 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-oauth-config\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:59.392796 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:59.392672 6524 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered Nov 03 21:45:59.392796 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:59.392683 6524 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered Nov 03 21:45:59.392796 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:59.392704 6524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-oauth-config podName:866ddb5a-c929-40c1-911d-ca40c5405723 nodeName:}" failed. No retries permitted until 2026-11-03 21:46:03.39269739 +0000 UTC m=+8.745024216 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-oauth-config") pod "console-86cf959768-tf4fd" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723") : object "openshift-console"/"console-oauth-config" not registered Nov 03 21:45:59.392796 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:59.392769 6524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-trusted-ca-bundle podName:866ddb5a-c929-40c1-911d-ca40c5405723 nodeName:}" failed. No retries permitted until 2026-11-03 21:46:03.392756693 +0000 UTC m=+8.745083487 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-trusted-ca-bundle") pod "console-86cf959768-tf4fd" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723") : object "openshift-console"/"trusted-ca-bundle" not registered Nov 03 21:45:59.393378 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:59.392988 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-54674f497-2g5ct" Nov 03 21:45:59.493507 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:45:59.493470 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5kf4\" (UniqueName: \"kubernetes.io/projected/866ddb5a-c929-40c1-911d-ca40c5405723-kube-api-access-n5kf4\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:45:59.493692 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:59.493619 6524 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered Nov 03 21:45:59.493692 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:59.493647 6524 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered Nov 03 21:45:59.493692 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:59.493662 6524 projected.go:196] Error preparing data for projected volume kube-api-access-n5kf4 for pod openshift-console/console-86cf959768-tf4fd: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Nov 03 21:45:59.493859 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:59.493712 6524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/866ddb5a-c929-40c1-911d-ca40c5405723-kube-api-access-n5kf4 podName:866ddb5a-c929-40c1-911d-ca40c5405723 nodeName:}" failed. No retries permitted until 2026-11-03 21:46:03.493699241 +0000 UTC m=+8.846026022 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-n5kf4" (UniqueName: "kubernetes.io/projected/866ddb5a-c929-40c1-911d-ca40c5405723-kube-api-access-n5kf4") pod "console-86cf959768-tf4fd" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Nov 03 21:45:59.707163 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:45:59.707063 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 21:46:00.341617 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:00.341580 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-zrbsd" Nov 03 21:46:01.741364 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:01.741331 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-9cnqh" Nov 03 21:46:03.434160 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:03.434100 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-serving-cert\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:46:03.434160 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:03.434160 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-console-config\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:46:03.434783 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:03.434183 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-oauth-config\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:46:03.434783 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:03.434219 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-service-ca\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:46:03.434783 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:03.434254 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-oauth-serving-cert\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:46:03.434783 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:03.434280 6524 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered Nov 03 21:46:03.434783 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:03.434291 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-trusted-ca-bundle\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:46:03.434783 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:03.434304 6524 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered Nov 03 21:46:03.434783 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:03.434323 6524 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered Nov 03 21:46:03.434783 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:03.434356 6524 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered Nov 03 21:46:03.434783 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:03.434389 6524 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered Nov 03 21:46:03.434783 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:03.434367 6524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-serving-cert podName:866ddb5a-c929-40c1-911d-ca40c5405723 nodeName:}" failed. No retries permitted until 2026-11-03 21:46:11.434345365 +0000 UTC m=+16.786672161 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-serving-cert") pod "console-86cf959768-tf4fd" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723") : object "openshift-console"/"console-serving-cert" not registered Nov 03 21:46:03.434783 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:03.434408 6524 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered Nov 03 21:46:03.434783 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:03.434439 6524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-console-config podName:866ddb5a-c929-40c1-911d-ca40c5405723 nodeName:}" failed. No retries permitted until 2026-11-03 21:46:11.434420042 +0000 UTC m=+16.786746825 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-console-config") pod "console-86cf959768-tf4fd" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723") : object "openshift-console"/"console-config" not registered Nov 03 21:46:03.434783 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:03.434457 6524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-service-ca podName:866ddb5a-c929-40c1-911d-ca40c5405723 nodeName:}" failed. No retries permitted until 2026-11-03 21:46:11.434447046 +0000 UTC m=+16.786773828 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-service-ca") pod "console-86cf959768-tf4fd" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723") : object "openshift-console"/"service-ca" not registered Nov 03 21:46:03.434783 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:03.434479 6524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-oauth-config podName:866ddb5a-c929-40c1-911d-ca40c5405723 nodeName:}" failed. No retries permitted until 2026-11-03 21:46:11.434469708 +0000 UTC m=+16.786796494 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-oauth-config") pod "console-86cf959768-tf4fd" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723") : object "openshift-console"/"console-oauth-config" not registered Nov 03 21:46:03.434783 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:03.434503 6524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-oauth-serving-cert podName:866ddb5a-c929-40c1-911d-ca40c5405723 nodeName:}" failed. No retries permitted until 2026-11-03 21:46:11.434489284 +0000 UTC m=+16.786816080 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-oauth-serving-cert") pod "console-86cf959768-tf4fd" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723") : object "openshift-console"/"oauth-serving-cert" not registered Nov 03 21:46:03.434783 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:03.434516 6524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-trusted-ca-bundle podName:866ddb5a-c929-40c1-911d-ca40c5405723 nodeName:}" failed. No retries permitted until 2026-11-03 21:46:11.434510078 +0000 UTC m=+16.786836860 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-trusted-ca-bundle") pod "console-86cf959768-tf4fd" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723") : object "openshift-console"/"trusted-ca-bundle" not registered Nov 03 21:46:03.535525 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:03.535482 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5kf4\" (UniqueName: \"kubernetes.io/projected/866ddb5a-c929-40c1-911d-ca40c5405723-kube-api-access-n5kf4\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:46:03.535712 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:03.535659 6524 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered Nov 03 21:46:03.535712 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:03.535684 6524 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered Nov 03 21:46:03.535712 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:03.535693 6524 projected.go:196] Error preparing data for projected volume kube-api-access-n5kf4 for pod openshift-console/console-86cf959768-tf4fd: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Nov 03 21:46:03.535836 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:03.535750 6524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/866ddb5a-c929-40c1-911d-ca40c5405723-kube-api-access-n5kf4 podName:866ddb5a-c929-40c1-911d-ca40c5405723 nodeName:}" failed. No retries permitted until 2026-11-03 21:46:11.53573251 +0000 UTC m=+16.888059310 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-n5kf4" (UniqueName: "kubernetes.io/projected/866ddb5a-c929-40c1-911d-ca40c5405723-kube-api-access-n5kf4") pod "console-86cf959768-tf4fd" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Nov 03 21:46:11.514413 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:11.514311 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-service-ca\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:46:11.514413 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:11.514363 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-oauth-serving-cert\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:46:11.514413 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:11.514392 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-trusted-ca-bundle\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:46:11.515032 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:11.514458 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-serving-cert\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:46:11.515032 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:11.514483 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-console-config\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:46:11.515032 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:11.514510 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-oauth-config\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:46:11.515032 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:11.514458 6524 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered Nov 03 21:46:11.515032 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:11.514582 6524 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered Nov 03 21:46:11.515032 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:11.514593 6524 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered Nov 03 21:46:11.515032 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:11.514632 6524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-oauth-serving-cert podName:866ddb5a-c929-40c1-911d-ca40c5405723 nodeName:}" failed. No retries permitted until 2026-11-03 21:46:27.514606551 +0000 UTC m=+32.866933353 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-oauth-serving-cert") pod "console-86cf959768-tf4fd" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723") : object "openshift-console"/"oauth-serving-cert" not registered Nov 03 21:46:11.515032 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:11.514458 6524 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered Nov 03 21:46:11.515032 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:11.514654 6524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-console-config podName:866ddb5a-c929-40c1-911d-ca40c5405723 nodeName:}" failed. No retries permitted until 2026-11-03 21:46:27.514643688 +0000 UTC m=+32.866970479 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-console-config") pod "console-86cf959768-tf4fd" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723") : object "openshift-console"/"console-config" not registered Nov 03 21:46:11.515032 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:11.514485 6524 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered Nov 03 21:46:11.515032 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:11.514671 6524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-oauth-config podName:866ddb5a-c929-40c1-911d-ca40c5405723 nodeName:}" failed. No retries permitted until 2026-11-03 21:46:27.514661315 +0000 UTC m=+32.866988099 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-oauth-config") pod "console-86cf959768-tf4fd" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723") : object "openshift-console"/"console-oauth-config" not registered Nov 03 21:46:11.515032 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:11.514686 6524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-service-ca podName:866ddb5a-c929-40c1-911d-ca40c5405723 nodeName:}" failed. No retries permitted until 2026-11-03 21:46:27.51467794 +0000 UTC m=+32.867004741 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-service-ca") pod "console-86cf959768-tf4fd" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723") : object "openshift-console"/"service-ca" not registered Nov 03 21:46:11.515032 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:11.514538 6524 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered Nov 03 21:46:11.515032 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:11.514725 6524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-trusted-ca-bundle podName:866ddb5a-c929-40c1-911d-ca40c5405723 nodeName:}" failed. No retries permitted until 2026-11-03 21:46:27.514704225 +0000 UTC m=+32.867031024 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-trusted-ca-bundle") pod "console-86cf959768-tf4fd" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723") : object "openshift-console"/"trusted-ca-bundle" not registered Nov 03 21:46:11.515032 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:11.514761 6524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-serving-cert podName:866ddb5a-c929-40c1-911d-ca40c5405723 nodeName:}" failed. No retries permitted until 2026-11-03 21:46:27.514753446 +0000 UTC m=+32.867080228 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-serving-cert") pod "console-86cf959768-tf4fd" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723") : object "openshift-console"/"console-serving-cert" not registered Nov 03 21:46:11.615673 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:11.615634 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5kf4\" (UniqueName: \"kubernetes.io/projected/866ddb5a-c929-40c1-911d-ca40c5405723-kube-api-access-n5kf4\") pod \"console-86cf959768-tf4fd\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:46:11.615827 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:11.615805 6524 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered Nov 03 21:46:11.615874 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:11.615833 6524 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered Nov 03 21:46:11.615874 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:11.615844 6524 projected.go:196] Error preparing data for projected volume kube-api-access-n5kf4 for pod openshift-console/console-86cf959768-tf4fd: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Nov 03 21:46:11.615939 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:11.615898 6524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/866ddb5a-c929-40c1-911d-ca40c5405723-kube-api-access-n5kf4 podName:866ddb5a-c929-40c1-911d-ca40c5405723 nodeName:}" failed. No retries permitted until 2026-11-03 21:46:27.61588243 +0000 UTC m=+32.968209229 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-n5kf4" (UniqueName: "kubernetes.io/projected/866ddb5a-c929-40c1-911d-ca40c5405723-kube-api-access-n5kf4") pod "console-86cf959768-tf4fd" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered] Nov 03 21:46:14.933505 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:14.933474 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-v98zn" Nov 03 21:46:18.160574 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:18.160541 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 03 21:46:18.161080 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:18.160957 6524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" containerName="alertmanager" containerID="cri-o://fac668a5bc5de0acff68159a1e9e2a066fec806a7736cd929c31307f799c04a3" gracePeriod=120 Nov 03 21:46:18.161080 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:18.161034 6524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" containerName="prom-label-proxy" containerID="cri-o://1ac717af4b9b79b11ecbbb6d351a9e05b4d7f1c903b2164fa001e3a33863d719" gracePeriod=120 Nov 03 21:46:18.161080 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:18.161020 6524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" containerName="kube-rbac-proxy-metric" containerID="cri-o://ff688c729e043f4e44ad64cdf30845d30351c59caca4b5c995947f01364693aa" gracePeriod=120 Nov 03 21:46:18.161289 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:18.161071 6524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" containerName="kube-rbac-proxy-web" containerID="cri-o://4c9079c6c502168773c81a19b51f0fc5c0f96e4788ddc40dbe93dc4e907bda46" gracePeriod=120 Nov 03 21:46:18.161289 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:18.161083 6524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" containerName="kube-rbac-proxy" containerID="cri-o://b644bdeff48ac3f9a1a191846ea60b2dc37dbb46b3b0d14c78af59193858685e" gracePeriod=120 Nov 03 21:46:18.161289 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:18.161089 6524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" containerName="config-reloader" containerID="cri-o://58b5804393fb6229f77781e0eb3ddcaaa735b2844861751aec958351c04fc520" gracePeriod=120 Nov 03 21:46:18.914596 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:18.914563 6524 generic.go:356] "Generic (PLEG): container finished" podID="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" containerID="1ac717af4b9b79b11ecbbb6d351a9e05b4d7f1c903b2164fa001e3a33863d719" exitCode=0 Nov 03 21:46:18.914596 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:18.914591 6524 generic.go:356] "Generic (PLEG): container finished" podID="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" containerID="b644bdeff48ac3f9a1a191846ea60b2dc37dbb46b3b0d14c78af59193858685e" exitCode=0 Nov 03 21:46:18.914596 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:18.914598 6524 generic.go:356] "Generic (PLEG): container finished" podID="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" containerID="4c9079c6c502168773c81a19b51f0fc5c0f96e4788ddc40dbe93dc4e907bda46" exitCode=0 Nov 03 21:46:18.914836 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:18.914607 6524 generic.go:356] "Generic (PLEG): container finished" podID="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" containerID="58b5804393fb6229f77781e0eb3ddcaaa735b2844861751aec958351c04fc520" exitCode=0 Nov 03 21:46:18.914836 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:18.914619 6524 generic.go:356] "Generic (PLEG): container finished" podID="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" containerID="fac668a5bc5de0acff68159a1e9e2a066fec806a7736cd929c31307f799c04a3" exitCode=0 Nov 03 21:46:18.914836 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:18.914632 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a2ccde5e-94a2-47ac-aff9-65248e4a6c92","Type":"ContainerDied","Data":"1ac717af4b9b79b11ecbbb6d351a9e05b4d7f1c903b2164fa001e3a33863d719"} Nov 03 21:46:18.914836 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:18.914710 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a2ccde5e-94a2-47ac-aff9-65248e4a6c92","Type":"ContainerDied","Data":"b644bdeff48ac3f9a1a191846ea60b2dc37dbb46b3b0d14c78af59193858685e"} Nov 03 21:46:18.914836 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:18.914737 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a2ccde5e-94a2-47ac-aff9-65248e4a6c92","Type":"ContainerDied","Data":"4c9079c6c502168773c81a19b51f0fc5c0f96e4788ddc40dbe93dc4e907bda46"} Nov 03 21:46:18.914836 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:18.914761 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a2ccde5e-94a2-47ac-aff9-65248e4a6c92","Type":"ContainerDied","Data":"58b5804393fb6229f77781e0eb3ddcaaa735b2844861751aec958351c04fc520"} Nov 03 21:46:18.914836 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:18.914784 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a2ccde5e-94a2-47ac-aff9-65248e4a6c92","Type":"ContainerDied","Data":"fac668a5bc5de0acff68159a1e9e2a066fec806a7736cd929c31307f799c04a3"} Nov 03 21:46:19.404944 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.404917 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:46:19.486905 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.486824 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-secret-alertmanager-kube-rbac-proxy\") pod \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " Nov 03 21:46:19.486905 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.486890 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-cluster-tls-config\") pod \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " Nov 03 21:46:19.487095 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.486979 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-alertmanager-main-db\") pod \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " Nov 03 21:46:19.487095 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.487012 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-tls-assets\" (UniqueName: \"kubernetes.io/projected/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-tls-assets\") pod \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " Nov 03 21:46:19.487095 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.487050 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-metrics-client-ca\") pod \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " Nov 03 21:46:19.487095 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.487084 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-config-out\" (UniqueName: \"kubernetes.io/empty-dir/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-config-out\") pod \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " Nov 03 21:46:19.487270 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.487150 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-secret-alertmanager-kube-rbac-proxy-metric\") pod \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " Nov 03 21:46:19.487270 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.487180 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-web-config\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-web-config\") pod \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " Nov 03 21:46:19.487270 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.487213 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-secret-alertmanager-main-tls\") pod \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " Nov 03 21:46:19.487417 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.487279 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-secret-alertmanager-kube-rbac-proxy-web\") pod \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " Nov 03 21:46:19.487417 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.487323 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-config-volume\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-config-volume\") pod \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " Nov 03 21:46:19.487417 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.487366 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-alertmanager-trusted-ca-bundle\") pod \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " Nov 03 21:46:19.487417 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.487401 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-kube-api-access-s7g58\" (UniqueName: \"kubernetes.io/projected/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-kube-api-access-s7g58\") pod \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\" (UID: \"a2ccde5e-94a2-47ac-aff9-65248e4a6c92\") " Nov 03 21:46:19.487692 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.487514 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-metrics-client-ca" pod "a2ccde5e-94a2-47ac-aff9-65248e4a6c92" (UID: "a2ccde5e-94a2-47ac-aff9-65248e4a6c92"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 03 21:46:19.487692 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.487627 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-alertmanager-main-db" pod "a2ccde5e-94a2-47ac-aff9-65248e4a6c92" (UID: "a2ccde5e-94a2-47ac-aff9-65248e4a6c92"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 21:46:19.488390 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.487823 6524 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-alertmanager-main-db\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:46:19.488390 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.487851 6524 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-metrics-client-ca\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:46:19.488902 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.488873 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-alertmanager-trusted-ca-bundle" pod "a2ccde5e-94a2-47ac-aff9-65248e4a6c92" (UID: "a2ccde5e-94a2-47ac-aff9-65248e4a6c92"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 03 21:46:19.490616 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.490573 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-secret-alertmanager-kube-rbac-proxy" pod "a2ccde5e-94a2-47ac-aff9-65248e4a6c92" (UID: "a2ccde5e-94a2-47ac-aff9-65248e4a6c92"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 03 21:46:19.490943 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.490913 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-config-out" pod "a2ccde5e-94a2-47ac-aff9-65248e4a6c92" (UID: "a2ccde5e-94a2-47ac-aff9-65248e4a6c92"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 21:46:19.490943 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.490937 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-kube-api-access-s7g58" pod "a2ccde5e-94a2-47ac-aff9-65248e4a6c92" (UID: "a2ccde5e-94a2-47ac-aff9-65248e4a6c92"). InnerVolumeSpecName "kube-api-access-s7g58". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 03 21:46:19.492351 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.492312 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-secret-alertmanager-kube-rbac-proxy-metric" pod "a2ccde5e-94a2-47ac-aff9-65248e4a6c92" (UID: "a2ccde5e-94a2-47ac-aff9-65248e4a6c92"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 03 21:46:19.492509 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.492483 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-tls-assets" pod "a2ccde5e-94a2-47ac-aff9-65248e4a6c92" (UID: "a2ccde5e-94a2-47ac-aff9-65248e4a6c92"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 03 21:46:19.492585 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.492519 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-secret-alertmanager-kube-rbac-proxy-web" pod "a2ccde5e-94a2-47ac-aff9-65248e4a6c92" (UID: "a2ccde5e-94a2-47ac-aff9-65248e4a6c92"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 03 21:46:19.492776 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.492749 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-config-volume" pod "a2ccde5e-94a2-47ac-aff9-65248e4a6c92" (UID: "a2ccde5e-94a2-47ac-aff9-65248e4a6c92"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 03 21:46:19.492884 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.492864 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-secret-alertmanager-main-tls" pod "a2ccde5e-94a2-47ac-aff9-65248e4a6c92" (UID: "a2ccde5e-94a2-47ac-aff9-65248e4a6c92"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 03 21:46:19.496208 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.496180 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-cluster-tls-config" pod "a2ccde5e-94a2-47ac-aff9-65248e4a6c92" (UID: "a2ccde5e-94a2-47ac-aff9-65248e4a6c92"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 03 21:46:19.501905 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.501876 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-web-config" pod "a2ccde5e-94a2-47ac-aff9-65248e4a6c92" (UID: "a2ccde5e-94a2-47ac-aff9-65248e4a6c92"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 03 21:46:19.589233 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.589186 6524 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-config-out\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:46:19.589233 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.589227 6524 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:46:19.589233 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.589238 6524 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-web-config\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:46:19.589233 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.589247 6524 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-secret-alertmanager-main-tls\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:46:19.589233 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.589256 6524 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:46:19.589539 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.589265 6524 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-config-volume\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:46:19.589539 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.589274 6524 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:46:19.589539 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.589283 6524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s7g58\" (UniqueName: \"kubernetes.io/projected/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-kube-api-access-s7g58\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:46:19.589539 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.589293 6524 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:46:19.589539 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.589301 6524 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-cluster-tls-config\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:46:19.589539 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.589310 6524 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a2ccde5e-94a2-47ac-aff9-65248e4a6c92-tls-assets\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:46:19.921755 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.921724 6524 generic.go:356] "Generic (PLEG): container finished" podID="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" containerID="ff688c729e043f4e44ad64cdf30845d30351c59caca4b5c995947f01364693aa" exitCode=0 Nov 03 21:46:19.921920 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.921781 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a2ccde5e-94a2-47ac-aff9-65248e4a6c92","Type":"ContainerDied","Data":"ff688c729e043f4e44ad64cdf30845d30351c59caca4b5c995947f01364693aa"} Nov 03 21:46:19.921920 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.921817 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Nov 03 21:46:19.921920 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.921836 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a2ccde5e-94a2-47ac-aff9-65248e4a6c92","Type":"ContainerDied","Data":"106c43f9fcf1a1c3acc5d75e8010b45678a9a16af360e8ab411f4c897fd8dbc2"} Nov 03 21:46:19.921920 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.921870 6524 scope.go:122] "RemoveContainer" containerID="1ac717af4b9b79b11ecbbb6d351a9e05b4d7f1c903b2164fa001e3a33863d719" Nov 03 21:46:19.930231 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.930209 6524 scope.go:122] "RemoveContainer" containerID="ff688c729e043f4e44ad64cdf30845d30351c59caca4b5c995947f01364693aa" Nov 03 21:46:19.940354 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.940333 6524 scope.go:122] "RemoveContainer" containerID="b644bdeff48ac3f9a1a191846ea60b2dc37dbb46b3b0d14c78af59193858685e" Nov 03 21:46:19.947002 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.946985 6524 scope.go:122] "RemoveContainer" containerID="4c9079c6c502168773c81a19b51f0fc5c0f96e4788ddc40dbe93dc4e907bda46" Nov 03 21:46:19.950685 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.950665 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 03 21:46:19.954992 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.954967 6524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 03 21:46:19.955325 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.955300 6524 scope.go:122] "RemoveContainer" containerID="58b5804393fb6229f77781e0eb3ddcaaa735b2844861751aec958351c04fc520" Nov 03 21:46:19.962653 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.962621 6524 scope.go:122] "RemoveContainer" containerID="fac668a5bc5de0acff68159a1e9e2a066fec806a7736cd929c31307f799c04a3" Nov 03 21:46:19.969437 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.969413 6524 scope.go:122] "RemoveContainer" containerID="1d7e1a070f9e0542150676350d5dbd14e69951f115872bce24a88b74c9fb9a68" Nov 03 21:46:19.976419 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.976401 6524 scope.go:122] "RemoveContainer" containerID="1ac717af4b9b79b11ecbbb6d351a9e05b4d7f1c903b2164fa001e3a33863d719" Nov 03 21:46:19.976729 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:19.976706 6524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ac717af4b9b79b11ecbbb6d351a9e05b4d7f1c903b2164fa001e3a33863d719\": container with ID starting with 1ac717af4b9b79b11ecbbb6d351a9e05b4d7f1c903b2164fa001e3a33863d719 not found: ID does not exist" containerID="1ac717af4b9b79b11ecbbb6d351a9e05b4d7f1c903b2164fa001e3a33863d719" Nov 03 21:46:19.976773 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.976740 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ac717af4b9b79b11ecbbb6d351a9e05b4d7f1c903b2164fa001e3a33863d719"} err="failed to get container status \"1ac717af4b9b79b11ecbbb6d351a9e05b4d7f1c903b2164fa001e3a33863d719\": rpc error: code = NotFound desc = could not find container \"1ac717af4b9b79b11ecbbb6d351a9e05b4d7f1c903b2164fa001e3a33863d719\": container with ID starting with 1ac717af4b9b79b11ecbbb6d351a9e05b4d7f1c903b2164fa001e3a33863d719 not found: ID does not exist" Nov 03 21:46:19.976818 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.976775 6524 scope.go:122] "RemoveContainer" containerID="ff688c729e043f4e44ad64cdf30845d30351c59caca4b5c995947f01364693aa" Nov 03 21:46:19.977035 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:19.977018 6524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff688c729e043f4e44ad64cdf30845d30351c59caca4b5c995947f01364693aa\": container with ID starting with ff688c729e043f4e44ad64cdf30845d30351c59caca4b5c995947f01364693aa not found: ID does not exist" containerID="ff688c729e043f4e44ad64cdf30845d30351c59caca4b5c995947f01364693aa" Nov 03 21:46:19.977077 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.977042 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff688c729e043f4e44ad64cdf30845d30351c59caca4b5c995947f01364693aa"} err="failed to get container status \"ff688c729e043f4e44ad64cdf30845d30351c59caca4b5c995947f01364693aa\": rpc error: code = NotFound desc = could not find container \"ff688c729e043f4e44ad64cdf30845d30351c59caca4b5c995947f01364693aa\": container with ID starting with ff688c729e043f4e44ad64cdf30845d30351c59caca4b5c995947f01364693aa not found: ID does not exist" Nov 03 21:46:19.977077 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.977057 6524 scope.go:122] "RemoveContainer" containerID="b644bdeff48ac3f9a1a191846ea60b2dc37dbb46b3b0d14c78af59193858685e" Nov 03 21:46:19.977307 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:19.977290 6524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b644bdeff48ac3f9a1a191846ea60b2dc37dbb46b3b0d14c78af59193858685e\": container with ID starting with b644bdeff48ac3f9a1a191846ea60b2dc37dbb46b3b0d14c78af59193858685e not found: ID does not exist" containerID="b644bdeff48ac3f9a1a191846ea60b2dc37dbb46b3b0d14c78af59193858685e" Nov 03 21:46:19.977346 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.977313 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b644bdeff48ac3f9a1a191846ea60b2dc37dbb46b3b0d14c78af59193858685e"} err="failed to get container status \"b644bdeff48ac3f9a1a191846ea60b2dc37dbb46b3b0d14c78af59193858685e\": rpc error: code = NotFound desc = could not find container \"b644bdeff48ac3f9a1a191846ea60b2dc37dbb46b3b0d14c78af59193858685e\": container with ID starting with b644bdeff48ac3f9a1a191846ea60b2dc37dbb46b3b0d14c78af59193858685e not found: ID does not exist" Nov 03 21:46:19.977346 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.977328 6524 scope.go:122] "RemoveContainer" containerID="4c9079c6c502168773c81a19b51f0fc5c0f96e4788ddc40dbe93dc4e907bda46" Nov 03 21:46:19.977560 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:19.977544 6524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c9079c6c502168773c81a19b51f0fc5c0f96e4788ddc40dbe93dc4e907bda46\": container with ID starting with 4c9079c6c502168773c81a19b51f0fc5c0f96e4788ddc40dbe93dc4e907bda46 not found: ID does not exist" containerID="4c9079c6c502168773c81a19b51f0fc5c0f96e4788ddc40dbe93dc4e907bda46" Nov 03 21:46:19.977593 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.977565 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c9079c6c502168773c81a19b51f0fc5c0f96e4788ddc40dbe93dc4e907bda46"} err="failed to get container status \"4c9079c6c502168773c81a19b51f0fc5c0f96e4788ddc40dbe93dc4e907bda46\": rpc error: code = NotFound desc = could not find container \"4c9079c6c502168773c81a19b51f0fc5c0f96e4788ddc40dbe93dc4e907bda46\": container with ID starting with 4c9079c6c502168773c81a19b51f0fc5c0f96e4788ddc40dbe93dc4e907bda46 not found: ID does not exist" Nov 03 21:46:19.977593 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.977579 6524 scope.go:122] "RemoveContainer" containerID="58b5804393fb6229f77781e0eb3ddcaaa735b2844861751aec958351c04fc520" Nov 03 21:46:19.977793 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:19.977777 6524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58b5804393fb6229f77781e0eb3ddcaaa735b2844861751aec958351c04fc520\": container with ID starting with 58b5804393fb6229f77781e0eb3ddcaaa735b2844861751aec958351c04fc520 not found: ID does not exist" containerID="58b5804393fb6229f77781e0eb3ddcaaa735b2844861751aec958351c04fc520" Nov 03 21:46:19.977835 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.977799 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58b5804393fb6229f77781e0eb3ddcaaa735b2844861751aec958351c04fc520"} err="failed to get container status \"58b5804393fb6229f77781e0eb3ddcaaa735b2844861751aec958351c04fc520\": rpc error: code = NotFound desc = could not find container \"58b5804393fb6229f77781e0eb3ddcaaa735b2844861751aec958351c04fc520\": container with ID starting with 58b5804393fb6229f77781e0eb3ddcaaa735b2844861751aec958351c04fc520 not found: ID does not exist" Nov 03 21:46:19.977835 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.977813 6524 scope.go:122] "RemoveContainer" containerID="fac668a5bc5de0acff68159a1e9e2a066fec806a7736cd929c31307f799c04a3" Nov 03 21:46:19.978023 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:19.978009 6524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fac668a5bc5de0acff68159a1e9e2a066fec806a7736cd929c31307f799c04a3\": container with ID starting with fac668a5bc5de0acff68159a1e9e2a066fec806a7736cd929c31307f799c04a3 not found: ID does not exist" containerID="fac668a5bc5de0acff68159a1e9e2a066fec806a7736cd929c31307f799c04a3" Nov 03 21:46:19.978073 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.978026 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fac668a5bc5de0acff68159a1e9e2a066fec806a7736cd929c31307f799c04a3"} err="failed to get container status \"fac668a5bc5de0acff68159a1e9e2a066fec806a7736cd929c31307f799c04a3\": rpc error: code = NotFound desc = could not find container \"fac668a5bc5de0acff68159a1e9e2a066fec806a7736cd929c31307f799c04a3\": container with ID starting with fac668a5bc5de0acff68159a1e9e2a066fec806a7736cd929c31307f799c04a3 not found: ID does not exist" Nov 03 21:46:19.978073 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.978040 6524 scope.go:122] "RemoveContainer" containerID="1d7e1a070f9e0542150676350d5dbd14e69951f115872bce24a88b74c9fb9a68" Nov 03 21:46:19.978251 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:19.978236 6524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d7e1a070f9e0542150676350d5dbd14e69951f115872bce24a88b74c9fb9a68\": container with ID starting with 1d7e1a070f9e0542150676350d5dbd14e69951f115872bce24a88b74c9fb9a68 not found: ID does not exist" containerID="1d7e1a070f9e0542150676350d5dbd14e69951f115872bce24a88b74c9fb9a68" Nov 03 21:46:19.978296 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:19.978254 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d7e1a070f9e0542150676350d5dbd14e69951f115872bce24a88b74c9fb9a68"} err="failed to get container status \"1d7e1a070f9e0542150676350d5dbd14e69951f115872bce24a88b74c9fb9a68\": rpc error: code = NotFound desc = could not find container \"1d7e1a070f9e0542150676350d5dbd14e69951f115872bce24a88b74c9fb9a68\": container with ID starting with 1d7e1a070f9e0542150676350d5dbd14e69951f115872bce24a88b74c9fb9a68 not found: ID does not exist" Nov 03 21:46:20.763947 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:20.763909 6524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-86cf959768-tf4fd" podUID="866ddb5a-c929-40c1-911d-ca40c5405723" containerName="console" containerID="cri-o://777c53c0c6f5e7620a73772d2a60d00f6eb82ad24cbe6a2cb344b1bc9f4ebcde" gracePeriod=15 Nov 03 21:46:20.766358 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:20.766335 6524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" path="/var/lib/kubelet/pods/a2ccde5e-94a2-47ac-aff9-65248e4a6c92/volumes" Nov 03 21:46:20.929079 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:20.929046 6524 generic.go:356] "Generic (PLEG): container finished" podID="866ddb5a-c929-40c1-911d-ca40c5405723" containerID="777c53c0c6f5e7620a73772d2a60d00f6eb82ad24cbe6a2cb344b1bc9f4ebcde" exitCode=0 Nov 03 21:46:20.929242 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:20.929173 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86cf959768-tf4fd" event={"ID":"866ddb5a-c929-40c1-911d-ca40c5405723","Type":"ContainerDied","Data":"777c53c0c6f5e7620a73772d2a60d00f6eb82ad24cbe6a2cb344b1bc9f4ebcde"} Nov 03 21:46:21.010463 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:21.010441 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:46:21.103475 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:21.103438 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-service-ca\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-service-ca\") pod \"866ddb5a-c929-40c1-911d-ca40c5405723\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " Nov 03 21:46:21.103623 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:21.103507 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-serving-cert\") pod \"866ddb5a-c929-40c1-911d-ca40c5405723\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " Nov 03 21:46:21.103623 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:21.103531 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-oauth-serving-cert\") pod \"866ddb5a-c929-40c1-911d-ca40c5405723\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " Nov 03 21:46:21.103623 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:21.103558 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/866ddb5a-c929-40c1-911d-ca40c5405723-tmp\" (UniqueName: \"kubernetes.io/empty-dir/866ddb5a-c929-40c1-911d-ca40c5405723-tmp\") pod \"866ddb5a-c929-40c1-911d-ca40c5405723\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " Nov 03 21:46:21.103623 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:21.103583 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-oauth-config\") pod \"866ddb5a-c929-40c1-911d-ca40c5405723\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " Nov 03 21:46:21.103623 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:21.103613 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-console-config\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-console-config\") pod \"866ddb5a-c929-40c1-911d-ca40c5405723\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " Nov 03 21:46:21.103849 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:21.103641 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/866ddb5a-c929-40c1-911d-ca40c5405723-kube-api-access-n5kf4\" (UniqueName: \"kubernetes.io/projected/866ddb5a-c929-40c1-911d-ca40c5405723-kube-api-access-n5kf4\") pod \"866ddb5a-c929-40c1-911d-ca40c5405723\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " Nov 03 21:46:21.103849 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:21.103683 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-trusted-ca-bundle\") pod \"866ddb5a-c929-40c1-911d-ca40c5405723\" (UID: \"866ddb5a-c929-40c1-911d-ca40c5405723\") " Nov 03 21:46:21.103941 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:21.103885 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/866ddb5a-c929-40c1-911d-ca40c5405723-tmp" pod "866ddb5a-c929-40c1-911d-ca40c5405723" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 21:46:21.104026 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:21.103991 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-service-ca" pod "866ddb5a-c929-40c1-911d-ca40c5405723" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 03 21:46:21.104099 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:21.104068 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-oauth-serving-cert" pod "866ddb5a-c929-40c1-911d-ca40c5405723" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 03 21:46:21.104225 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:21.104203 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-console-config" pod "866ddb5a-c929-40c1-911d-ca40c5405723" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 03 21:46:21.104225 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:21.104215 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-trusted-ca-bundle" pod "866ddb5a-c929-40c1-911d-ca40c5405723" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 03 21:46:21.106180 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:21.106153 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-serving-cert" pod "866ddb5a-c929-40c1-911d-ca40c5405723" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 03 21:46:21.106288 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:21.106193 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/866ddb5a-c929-40c1-911d-ca40c5405723-kube-api-access-n5kf4" pod "866ddb5a-c929-40c1-911d-ca40c5405723" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723"). InnerVolumeSpecName "kube-api-access-n5kf4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 03 21:46:21.106288 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:21.106197 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-oauth-config" pod "866ddb5a-c929-40c1-911d-ca40c5405723" (UID: "866ddb5a-c929-40c1-911d-ca40c5405723"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 03 21:46:21.204825 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:21.204785 6524 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-oauth-serving-cert\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:46:21.204825 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:21.204817 6524 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/866ddb5a-c929-40c1-911d-ca40c5405723-tmp\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:46:21.204825 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:21.204828 6524 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-oauth-config\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:46:21.205062 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:21.204837 6524 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-console-config\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:46:21.205062 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:21.204847 6524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5kf4\" (UniqueName: \"kubernetes.io/projected/866ddb5a-c929-40c1-911d-ca40c5405723-kube-api-access-n5kf4\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:46:21.205062 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:21.204856 6524 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-trusted-ca-bundle\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:46:21.205062 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:21.204864 6524 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/866ddb5a-c929-40c1-911d-ca40c5405723-service-ca\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:46:21.205062 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:21.204873 6524 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/866ddb5a-c929-40c1-911d-ca40c5405723-console-serving-cert\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:46:21.933448 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:21.933413 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86cf959768-tf4fd" event={"ID":"866ddb5a-c929-40c1-911d-ca40c5405723","Type":"ContainerDied","Data":"ecfdcbf875981eb0a48d60cbc1003e95e865da3fd3fd6dec53c35397b6308e43"} Nov 03 21:46:21.933448 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:21.933433 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86cf959768-tf4fd" Nov 03 21:46:21.933982 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:21.933471 6524 scope.go:122] "RemoveContainer" containerID="777c53c0c6f5e7620a73772d2a60d00f6eb82ad24cbe6a2cb344b1bc9f4ebcde" Nov 03 21:46:21.956281 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:21.956247 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86cf959768-tf4fd"] Nov 03 21:46:21.960287 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:21.960258 6524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-86cf959768-tf4fd"] Nov 03 21:46:22.416953 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.416913 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 03 21:46:22.417418 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.417375 6524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" containerName="prometheus" containerID="cri-o://1f9121440b907bd547107b875553716b36cf846bf8b11f652a22435d05b362ab" gracePeriod=600 Nov 03 21:46:22.417544 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.417404 6524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" containerName="kube-rbac-proxy" containerID="cri-o://9d5d3e4daab369ff5f40ed2108677706b29a2250f185a7207a41b26939615efb" gracePeriod=600 Nov 03 21:46:22.417544 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.417482 6524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" containerName="kube-rbac-proxy-web" containerID="cri-o://50fd1fee4f8ba5e31f1b78656c421ef48ede3afd8e4f85f7fe26540d536d502f" gracePeriod=600 Nov 03 21:46:22.417544 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.417501 6524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" containerName="config-reloader" containerID="cri-o://8fc4b9b4b217355d6d2c046bd20b54b62a6fa33031b4812bbe555ec7d6771f8d" gracePeriod=600 Nov 03 21:46:22.417690 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.417401 6524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" containerName="thanos-sidecar" containerID="cri-o://64d36f5b414bd0de01d0058f60c37b3051fe2f67b3b5834c066ae2d742c4d487" gracePeriod=600 Nov 03 21:46:22.417690 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.417394 6524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" containerName="kube-rbac-proxy-thanos" containerID="cri-o://a049ac28ba7cf6a3af10022353c8a212963dfa04ff032c77d3a428dac1c83f0e" gracePeriod=600 Nov 03 21:46:22.657536 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.657509 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:22.716381 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.716291 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-metrics-client-certs\") pod \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " Nov 03 21:46:22.716381 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.716338 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-config\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-config\") pod \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " Nov 03 21:46:22.716381 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.716364 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-configmap-metrics-client-ca\") pod \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " Nov 03 21:46:22.716381 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.716381 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-web-config\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-web-config\") pod \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " Nov 03 21:46:22.716700 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.716407 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-k8s-rulefiles-0\") pod \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " Nov 03 21:46:22.716700 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.716434 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-kube-api-access-mlgsq\" (UniqueName: \"kubernetes.io/projected/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-kube-api-access-mlgsq\") pod \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " Nov 03 21:46:22.716700 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.716464 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-k8s-rulefiles-2\") pod \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " Nov 03 21:46:22.716700 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.716499 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-tls-assets\" (UniqueName: \"kubernetes.io/projected/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-tls-assets\") pod \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " Nov 03 21:46:22.716700 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.716534 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-trusted-ca-bundle\") pod \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " Nov 03 21:46:22.716700 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.716565 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-k8s-db\") pod \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " Nov 03 21:46:22.716700 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.716596 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " Nov 03 21:46:22.716700 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.716628 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-configmap-kubelet-serving-ca-bundle\") pod \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " Nov 03 21:46:22.716700 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.716665 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-prometheus-k8s-tls\") pod \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " Nov 03 21:46:22.716700 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.716694 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-grpc-tls\") pod \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " Nov 03 21:46:22.717199 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.716728 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-config-out\" (UniqueName: \"kubernetes.io/empty-dir/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-config-out\") pod \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " Nov 03 21:46:22.717199 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.716758 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-kube-rbac-proxy\") pod \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " Nov 03 21:46:22.717199 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.716752 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-configmap-metrics-client-ca" pod "1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" (UID: "1a5cbb44-3dbf-40a1-916b-71465d2dc2ea"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 03 21:46:22.717199 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.716799 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " Nov 03 21:46:22.717199 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.716851 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-thanos-prometheus-http-client-file\") pod \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " Nov 03 21:46:22.717199 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.716893 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-configmap-serving-certs-ca-bundle\") pod \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " Nov 03 21:46:22.717199 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.716921 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-k8s-rulefiles-1\") pod \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\" (UID: \"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea\") " Nov 03 21:46:22.717199 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.717159 6524 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-configmap-metrics-client-ca\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:46:22.717655 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.717267 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-k8s-rulefiles-2" pod "1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" (UID: "1a5cbb44-3dbf-40a1-916b-71465d2dc2ea"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 03 21:46:22.717655 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.717308 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-trusted-ca-bundle" pod "1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" (UID: "1a5cbb44-3dbf-40a1-916b-71465d2dc2ea"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 03 21:46:22.717655 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.717450 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-k8s-rulefiles-1" pod "1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" (UID: "1a5cbb44-3dbf-40a1-916b-71465d2dc2ea"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 03 21:46:22.717811 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.717739 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-configmap-kubelet-serving-ca-bundle" pod "1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" (UID: "1a5cbb44-3dbf-40a1-916b-71465d2dc2ea"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 03 21:46:22.718024 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.717928 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-k8s-db" pod "1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" (UID: "1a5cbb44-3dbf-40a1-916b-71465d2dc2ea"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 21:46:22.720611 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.720583 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-tls-assets" pod "1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" (UID: "1a5cbb44-3dbf-40a1-916b-71465d2dc2ea"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 03 21:46:22.720764 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.720496 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-prometheus-k8s-thanos-sidecar-tls" pod "1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" (UID: "1a5cbb44-3dbf-40a1-916b-71465d2dc2ea"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 03 21:46:22.720851 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.720785 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-configmap-serving-certs-ca-bundle" pod "1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" (UID: "1a5cbb44-3dbf-40a1-916b-71465d2dc2ea"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 03 21:46:22.720851 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.720797 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-k8s-rulefiles-0" pod "1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" (UID: "1a5cbb44-3dbf-40a1-916b-71465d2dc2ea"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 03 21:46:22.720973 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.720864 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-kube-rbac-proxy" pod "1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" (UID: "1a5cbb44-3dbf-40a1-916b-71465d2dc2ea"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 03 21:46:22.720973 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.720952 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-prometheus-k8s-tls" pod "1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" (UID: "1a5cbb44-3dbf-40a1-916b-71465d2dc2ea"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 03 21:46:22.721086 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.721028 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-config" pod "1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" (UID: "1a5cbb44-3dbf-40a1-916b-71465d2dc2ea"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 03 21:46:22.721238 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.721219 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-kube-api-access-mlgsq" pod "1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" (UID: "1a5cbb44-3dbf-40a1-916b-71465d2dc2ea"). InnerVolumeSpecName "kube-api-access-mlgsq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 03 21:46:22.721961 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.721922 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-metrics-client-certs" pod "1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" (UID: "1a5cbb44-3dbf-40a1-916b-71465d2dc2ea"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 03 21:46:22.722066 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.722047 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-grpc-tls" pod "1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" (UID: "1a5cbb44-3dbf-40a1-916b-71465d2dc2ea"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 03 21:46:22.722310 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.722290 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-config-out" pod "1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" (UID: "1a5cbb44-3dbf-40a1-916b-71465d2dc2ea"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 21:46:22.722478 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.722460 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-prometheus-k8s-kube-rbac-proxy-web" pod "1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" (UID: "1a5cbb44-3dbf-40a1-916b-71465d2dc2ea"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 03 21:46:22.722689 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.722664 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-thanos-prometheus-http-client-file" pod "1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" (UID: "1a5cbb44-3dbf-40a1-916b-71465d2dc2ea"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 03 21:46:22.732304 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.732272 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-web-config" pod "1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" (UID: "1a5cbb44-3dbf-40a1-916b-71465d2dc2ea"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 03 21:46:22.765672 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.765641 6524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="866ddb5a-c929-40c1-911d-ca40c5405723" path="/var/lib/kubelet/pods/866ddb5a-c929-40c1-911d-ca40c5405723/volumes" Nov 03 21:46:22.817879 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.817846 6524 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:46:22.817879 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.817876 6524 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-tls-assets\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:46:22.817879 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.817888 6524 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-trusted-ca-bundle\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:46:22.818101 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.817898 6524 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-k8s-db\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:46:22.818101 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.817908 6524 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:46:22.818101 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.817918 6524 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:46:22.818101 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.817926 6524 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-prometheus-k8s-tls\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:46:22.818101 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.817935 6524 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-grpc-tls\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:46:22.818101 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.817943 6524 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-config-out\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:46:22.818101 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.817951 6524 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-kube-rbac-proxy\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:46:22.818101 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.817959 6524 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:46:22.818101 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.817968 6524 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-thanos-prometheus-http-client-file\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:46:22.818101 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.817977 6524 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:46:22.818101 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.817986 6524 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:46:22.818101 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.817995 6524 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-secret-metrics-client-certs\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:46:22.818101 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.818004 6524 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-config\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:46:22.818101 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.818030 6524 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-web-config\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:46:22.818101 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.818040 6524 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:46:22.818101 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.818048 6524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mlgsq\" (UniqueName: \"kubernetes.io/projected/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea-kube-api-access-mlgsq\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:46:22.940213 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.940182 6524 generic.go:356] "Generic (PLEG): container finished" podID="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" containerID="a049ac28ba7cf6a3af10022353c8a212963dfa04ff032c77d3a428dac1c83f0e" exitCode=0 Nov 03 21:46:22.940213 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.940205 6524 generic.go:356] "Generic (PLEG): container finished" podID="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" containerID="9d5d3e4daab369ff5f40ed2108677706b29a2250f185a7207a41b26939615efb" exitCode=0 Nov 03 21:46:22.940213 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.940212 6524 generic.go:356] "Generic (PLEG): container finished" podID="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" containerID="50fd1fee4f8ba5e31f1b78656c421ef48ede3afd8e4f85f7fe26540d536d502f" exitCode=0 Nov 03 21:46:22.940213 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.940218 6524 generic.go:356] "Generic (PLEG): container finished" podID="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" containerID="64d36f5b414bd0de01d0058f60c37b3051fe2f67b3b5834c066ae2d742c4d487" exitCode=0 Nov 03 21:46:22.940213 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.940223 6524 generic.go:356] "Generic (PLEG): container finished" podID="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" containerID="8fc4b9b4b217355d6d2c046bd20b54b62a6fa33031b4812bbe555ec7d6771f8d" exitCode=0 Nov 03 21:46:22.940830 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.940231 6524 generic.go:356] "Generic (PLEG): container finished" podID="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" containerID="1f9121440b907bd547107b875553716b36cf846bf8b11f652a22435d05b362ab" exitCode=0 Nov 03 21:46:22.940830 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.940260 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea","Type":"ContainerDied","Data":"a049ac28ba7cf6a3af10022353c8a212963dfa04ff032c77d3a428dac1c83f0e"} Nov 03 21:46:22.940830 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.940279 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:22.940830 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.940330 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea","Type":"ContainerDied","Data":"9d5d3e4daab369ff5f40ed2108677706b29a2250f185a7207a41b26939615efb"} Nov 03 21:46:22.940830 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.940362 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea","Type":"ContainerDied","Data":"50fd1fee4f8ba5e31f1b78656c421ef48ede3afd8e4f85f7fe26540d536d502f"} Nov 03 21:46:22.940830 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.940402 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea","Type":"ContainerDied","Data":"64d36f5b414bd0de01d0058f60c37b3051fe2f67b3b5834c066ae2d742c4d487"} Nov 03 21:46:22.940830 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.940429 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea","Type":"ContainerDied","Data":"8fc4b9b4b217355d6d2c046bd20b54b62a6fa33031b4812bbe555ec7d6771f8d"} Nov 03 21:46:22.940830 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.940403 6524 scope.go:122] "RemoveContainer" containerID="a049ac28ba7cf6a3af10022353c8a212963dfa04ff032c77d3a428dac1c83f0e" Nov 03 21:46:22.940830 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.940459 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea","Type":"ContainerDied","Data":"1f9121440b907bd547107b875553716b36cf846bf8b11f652a22435d05b362ab"} Nov 03 21:46:22.940830 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.940491 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1a5cbb44-3dbf-40a1-916b-71465d2dc2ea","Type":"ContainerDied","Data":"12d4118e93b819d9eb3d110e90fce86db0b42341bf9196e7600928553ea1077a"} Nov 03 21:46:22.948031 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.948014 6524 scope.go:122] "RemoveContainer" containerID="9d5d3e4daab369ff5f40ed2108677706b29a2250f185a7207a41b26939615efb" Nov 03 21:46:22.954696 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.954678 6524 scope.go:122] "RemoveContainer" containerID="50fd1fee4f8ba5e31f1b78656c421ef48ede3afd8e4f85f7fe26540d536d502f" Nov 03 21:46:22.961502 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.961484 6524 scope.go:122] "RemoveContainer" containerID="64d36f5b414bd0de01d0058f60c37b3051fe2f67b3b5834c066ae2d742c4d487" Nov 03 21:46:22.964605 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.964580 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 03 21:46:22.969112 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.968884 6524 scope.go:122] "RemoveContainer" containerID="8fc4b9b4b217355d6d2c046bd20b54b62a6fa33031b4812bbe555ec7d6771f8d" Nov 03 21:46:22.969560 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.969543 6524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 03 21:46:22.975584 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.975565 6524 scope.go:122] "RemoveContainer" containerID="1f9121440b907bd547107b875553716b36cf846bf8b11f652a22435d05b362ab" Nov 03 21:46:22.982472 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.982454 6524 scope.go:122] "RemoveContainer" containerID="14b9495cfb235a428a51ba26e6ccea26a7e4b5c67bbe3b4f43a6d924eff2decf" Nov 03 21:46:22.988678 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.988652 6524 scope.go:122] "RemoveContainer" containerID="a049ac28ba7cf6a3af10022353c8a212963dfa04ff032c77d3a428dac1c83f0e" Nov 03 21:46:22.988913 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:22.988894 6524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a049ac28ba7cf6a3af10022353c8a212963dfa04ff032c77d3a428dac1c83f0e\": container with ID starting with a049ac28ba7cf6a3af10022353c8a212963dfa04ff032c77d3a428dac1c83f0e not found: ID does not exist" containerID="a049ac28ba7cf6a3af10022353c8a212963dfa04ff032c77d3a428dac1c83f0e" Nov 03 21:46:22.988969 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.988923 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a049ac28ba7cf6a3af10022353c8a212963dfa04ff032c77d3a428dac1c83f0e"} err="failed to get container status \"a049ac28ba7cf6a3af10022353c8a212963dfa04ff032c77d3a428dac1c83f0e\": rpc error: code = NotFound desc = could not find container \"a049ac28ba7cf6a3af10022353c8a212963dfa04ff032c77d3a428dac1c83f0e\": container with ID starting with a049ac28ba7cf6a3af10022353c8a212963dfa04ff032c77d3a428dac1c83f0e not found: ID does not exist" Nov 03 21:46:22.988969 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.988945 6524 scope.go:122] "RemoveContainer" containerID="9d5d3e4daab369ff5f40ed2108677706b29a2250f185a7207a41b26939615efb" Nov 03 21:46:22.989232 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:22.989214 6524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d5d3e4daab369ff5f40ed2108677706b29a2250f185a7207a41b26939615efb\": container with ID starting with 9d5d3e4daab369ff5f40ed2108677706b29a2250f185a7207a41b26939615efb not found: ID does not exist" containerID="9d5d3e4daab369ff5f40ed2108677706b29a2250f185a7207a41b26939615efb" Nov 03 21:46:22.989293 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.989238 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d5d3e4daab369ff5f40ed2108677706b29a2250f185a7207a41b26939615efb"} err="failed to get container status \"9d5d3e4daab369ff5f40ed2108677706b29a2250f185a7207a41b26939615efb\": rpc error: code = NotFound desc = could not find container \"9d5d3e4daab369ff5f40ed2108677706b29a2250f185a7207a41b26939615efb\": container with ID starting with 9d5d3e4daab369ff5f40ed2108677706b29a2250f185a7207a41b26939615efb not found: ID does not exist" Nov 03 21:46:22.989293 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.989256 6524 scope.go:122] "RemoveContainer" containerID="50fd1fee4f8ba5e31f1b78656c421ef48ede3afd8e4f85f7fe26540d536d502f" Nov 03 21:46:22.989478 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:22.989460 6524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50fd1fee4f8ba5e31f1b78656c421ef48ede3afd8e4f85f7fe26540d536d502f\": container with ID starting with 50fd1fee4f8ba5e31f1b78656c421ef48ede3afd8e4f85f7fe26540d536d502f not found: ID does not exist" containerID="50fd1fee4f8ba5e31f1b78656c421ef48ede3afd8e4f85f7fe26540d536d502f" Nov 03 21:46:22.992791 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.989483 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50fd1fee4f8ba5e31f1b78656c421ef48ede3afd8e4f85f7fe26540d536d502f"} err="failed to get container status \"50fd1fee4f8ba5e31f1b78656c421ef48ede3afd8e4f85f7fe26540d536d502f\": rpc error: code = NotFound desc = could not find container \"50fd1fee4f8ba5e31f1b78656c421ef48ede3afd8e4f85f7fe26540d536d502f\": container with ID starting with 50fd1fee4f8ba5e31f1b78656c421ef48ede3afd8e4f85f7fe26540d536d502f not found: ID does not exist" Nov 03 21:46:22.992791 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.989812 6524 scope.go:122] "RemoveContainer" containerID="64d36f5b414bd0de01d0058f60c37b3051fe2f67b3b5834c066ae2d742c4d487" Nov 03 21:46:22.993365 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:22.993340 6524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64d36f5b414bd0de01d0058f60c37b3051fe2f67b3b5834c066ae2d742c4d487\": container with ID starting with 64d36f5b414bd0de01d0058f60c37b3051fe2f67b3b5834c066ae2d742c4d487 not found: ID does not exist" containerID="64d36f5b414bd0de01d0058f60c37b3051fe2f67b3b5834c066ae2d742c4d487" Nov 03 21:46:22.993440 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.993378 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64d36f5b414bd0de01d0058f60c37b3051fe2f67b3b5834c066ae2d742c4d487"} err="failed to get container status \"64d36f5b414bd0de01d0058f60c37b3051fe2f67b3b5834c066ae2d742c4d487\": rpc error: code = NotFound desc = could not find container \"64d36f5b414bd0de01d0058f60c37b3051fe2f67b3b5834c066ae2d742c4d487\": container with ID starting with 64d36f5b414bd0de01d0058f60c37b3051fe2f67b3b5834c066ae2d742c4d487 not found: ID does not exist" Nov 03 21:46:22.993440 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.993394 6524 scope.go:122] "RemoveContainer" containerID="8fc4b9b4b217355d6d2c046bd20b54b62a6fa33031b4812bbe555ec7d6771f8d" Nov 03 21:46:22.993653 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:22.993638 6524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fc4b9b4b217355d6d2c046bd20b54b62a6fa33031b4812bbe555ec7d6771f8d\": container with ID starting with 8fc4b9b4b217355d6d2c046bd20b54b62a6fa33031b4812bbe555ec7d6771f8d not found: ID does not exist" containerID="8fc4b9b4b217355d6d2c046bd20b54b62a6fa33031b4812bbe555ec7d6771f8d" Nov 03 21:46:22.993697 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.993656 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fc4b9b4b217355d6d2c046bd20b54b62a6fa33031b4812bbe555ec7d6771f8d"} err="failed to get container status \"8fc4b9b4b217355d6d2c046bd20b54b62a6fa33031b4812bbe555ec7d6771f8d\": rpc error: code = NotFound desc = could not find container \"8fc4b9b4b217355d6d2c046bd20b54b62a6fa33031b4812bbe555ec7d6771f8d\": container with ID starting with 8fc4b9b4b217355d6d2c046bd20b54b62a6fa33031b4812bbe555ec7d6771f8d not found: ID does not exist" Nov 03 21:46:22.993697 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.993694 6524 scope.go:122] "RemoveContainer" containerID="1f9121440b907bd547107b875553716b36cf846bf8b11f652a22435d05b362ab" Nov 03 21:46:22.993916 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:22.993898 6524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f9121440b907bd547107b875553716b36cf846bf8b11f652a22435d05b362ab\": container with ID starting with 1f9121440b907bd547107b875553716b36cf846bf8b11f652a22435d05b362ab not found: ID does not exist" containerID="1f9121440b907bd547107b875553716b36cf846bf8b11f652a22435d05b362ab" Nov 03 21:46:22.993976 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.993932 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f9121440b907bd547107b875553716b36cf846bf8b11f652a22435d05b362ab"} err="failed to get container status \"1f9121440b907bd547107b875553716b36cf846bf8b11f652a22435d05b362ab\": rpc error: code = NotFound desc = could not find container \"1f9121440b907bd547107b875553716b36cf846bf8b11f652a22435d05b362ab\": container with ID starting with 1f9121440b907bd547107b875553716b36cf846bf8b11f652a22435d05b362ab not found: ID does not exist" Nov 03 21:46:22.993976 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.993953 6524 scope.go:122] "RemoveContainer" containerID="14b9495cfb235a428a51ba26e6ccea26a7e4b5c67bbe3b4f43a6d924eff2decf" Nov 03 21:46:22.994282 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:22.994251 6524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14b9495cfb235a428a51ba26e6ccea26a7e4b5c67bbe3b4f43a6d924eff2decf\": container with ID starting with 14b9495cfb235a428a51ba26e6ccea26a7e4b5c67bbe3b4f43a6d924eff2decf not found: ID does not exist" containerID="14b9495cfb235a428a51ba26e6ccea26a7e4b5c67bbe3b4f43a6d924eff2decf" Nov 03 21:46:22.994342 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.994294 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14b9495cfb235a428a51ba26e6ccea26a7e4b5c67bbe3b4f43a6d924eff2decf"} err="failed to get container status \"14b9495cfb235a428a51ba26e6ccea26a7e4b5c67bbe3b4f43a6d924eff2decf\": rpc error: code = NotFound desc = could not find container \"14b9495cfb235a428a51ba26e6ccea26a7e4b5c67bbe3b4f43a6d924eff2decf\": container with ID starting with 14b9495cfb235a428a51ba26e6ccea26a7e4b5c67bbe3b4f43a6d924eff2decf not found: ID does not exist" Nov 03 21:46:22.994342 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.994322 6524 scope.go:122] "RemoveContainer" containerID="a049ac28ba7cf6a3af10022353c8a212963dfa04ff032c77d3a428dac1c83f0e" Nov 03 21:46:22.994713 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.994674 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a049ac28ba7cf6a3af10022353c8a212963dfa04ff032c77d3a428dac1c83f0e"} err="failed to get container status \"a049ac28ba7cf6a3af10022353c8a212963dfa04ff032c77d3a428dac1c83f0e\": rpc error: code = NotFound desc = could not find container \"a049ac28ba7cf6a3af10022353c8a212963dfa04ff032c77d3a428dac1c83f0e\": container with ID starting with a049ac28ba7cf6a3af10022353c8a212963dfa04ff032c77d3a428dac1c83f0e not found: ID does not exist" Nov 03 21:46:22.994780 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.994724 6524 scope.go:122] "RemoveContainer" containerID="9d5d3e4daab369ff5f40ed2108677706b29a2250f185a7207a41b26939615efb" Nov 03 21:46:22.995062 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.995042 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d5d3e4daab369ff5f40ed2108677706b29a2250f185a7207a41b26939615efb"} err="failed to get container status \"9d5d3e4daab369ff5f40ed2108677706b29a2250f185a7207a41b26939615efb\": rpc error: code = NotFound desc = could not find container \"9d5d3e4daab369ff5f40ed2108677706b29a2250f185a7207a41b26939615efb\": container with ID starting with 9d5d3e4daab369ff5f40ed2108677706b29a2250f185a7207a41b26939615efb not found: ID does not exist" Nov 03 21:46:22.995111 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.995061 6524 scope.go:122] "RemoveContainer" containerID="50fd1fee4f8ba5e31f1b78656c421ef48ede3afd8e4f85f7fe26540d536d502f" Nov 03 21:46:22.995304 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.995282 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50fd1fee4f8ba5e31f1b78656c421ef48ede3afd8e4f85f7fe26540d536d502f"} err="failed to get container status \"50fd1fee4f8ba5e31f1b78656c421ef48ede3afd8e4f85f7fe26540d536d502f\": rpc error: code = NotFound desc = could not find container \"50fd1fee4f8ba5e31f1b78656c421ef48ede3afd8e4f85f7fe26540d536d502f\": container with ID starting with 50fd1fee4f8ba5e31f1b78656c421ef48ede3afd8e4f85f7fe26540d536d502f not found: ID does not exist" Nov 03 21:46:22.995348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.995305 6524 scope.go:122] "RemoveContainer" containerID="64d36f5b414bd0de01d0058f60c37b3051fe2f67b3b5834c066ae2d742c4d487" Nov 03 21:46:22.995528 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.995513 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64d36f5b414bd0de01d0058f60c37b3051fe2f67b3b5834c066ae2d742c4d487"} err="failed to get container status \"64d36f5b414bd0de01d0058f60c37b3051fe2f67b3b5834c066ae2d742c4d487\": rpc error: code = NotFound desc = could not find container \"64d36f5b414bd0de01d0058f60c37b3051fe2f67b3b5834c066ae2d742c4d487\": container with ID starting with 64d36f5b414bd0de01d0058f60c37b3051fe2f67b3b5834c066ae2d742c4d487 not found: ID does not exist" Nov 03 21:46:22.995570 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.995528 6524 scope.go:122] "RemoveContainer" containerID="8fc4b9b4b217355d6d2c046bd20b54b62a6fa33031b4812bbe555ec7d6771f8d" Nov 03 21:46:22.995719 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.995702 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fc4b9b4b217355d6d2c046bd20b54b62a6fa33031b4812bbe555ec7d6771f8d"} err="failed to get container status \"8fc4b9b4b217355d6d2c046bd20b54b62a6fa33031b4812bbe555ec7d6771f8d\": rpc error: code = NotFound desc = could not find container \"8fc4b9b4b217355d6d2c046bd20b54b62a6fa33031b4812bbe555ec7d6771f8d\": container with ID starting with 8fc4b9b4b217355d6d2c046bd20b54b62a6fa33031b4812bbe555ec7d6771f8d not found: ID does not exist" Nov 03 21:46:22.995756 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.995720 6524 scope.go:122] "RemoveContainer" containerID="1f9121440b907bd547107b875553716b36cf846bf8b11f652a22435d05b362ab" Nov 03 21:46:22.995966 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.995949 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f9121440b907bd547107b875553716b36cf846bf8b11f652a22435d05b362ab"} err="failed to get container status \"1f9121440b907bd547107b875553716b36cf846bf8b11f652a22435d05b362ab\": rpc error: code = NotFound desc = could not find container \"1f9121440b907bd547107b875553716b36cf846bf8b11f652a22435d05b362ab\": container with ID starting with 1f9121440b907bd547107b875553716b36cf846bf8b11f652a22435d05b362ab not found: ID does not exist" Nov 03 21:46:22.996011 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.995966 6524 scope.go:122] "RemoveContainer" containerID="14b9495cfb235a428a51ba26e6ccea26a7e4b5c67bbe3b4f43a6d924eff2decf" Nov 03 21:46:22.996190 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.996171 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14b9495cfb235a428a51ba26e6ccea26a7e4b5c67bbe3b4f43a6d924eff2decf"} err="failed to get container status \"14b9495cfb235a428a51ba26e6ccea26a7e4b5c67bbe3b4f43a6d924eff2decf\": rpc error: code = NotFound desc = could not find container \"14b9495cfb235a428a51ba26e6ccea26a7e4b5c67bbe3b4f43a6d924eff2decf\": container with ID starting with 14b9495cfb235a428a51ba26e6ccea26a7e4b5c67bbe3b4f43a6d924eff2decf not found: ID does not exist" Nov 03 21:46:22.996238 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.996189 6524 scope.go:122] "RemoveContainer" containerID="a049ac28ba7cf6a3af10022353c8a212963dfa04ff032c77d3a428dac1c83f0e" Nov 03 21:46:22.996382 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.996364 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a049ac28ba7cf6a3af10022353c8a212963dfa04ff032c77d3a428dac1c83f0e"} err="failed to get container status \"a049ac28ba7cf6a3af10022353c8a212963dfa04ff032c77d3a428dac1c83f0e\": rpc error: code = NotFound desc = could not find container \"a049ac28ba7cf6a3af10022353c8a212963dfa04ff032c77d3a428dac1c83f0e\": container with ID starting with a049ac28ba7cf6a3af10022353c8a212963dfa04ff032c77d3a428dac1c83f0e not found: ID does not exist" Nov 03 21:46:22.996426 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.996383 6524 scope.go:122] "RemoveContainer" containerID="9d5d3e4daab369ff5f40ed2108677706b29a2250f185a7207a41b26939615efb" Nov 03 21:46:22.996610 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.996583 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d5d3e4daab369ff5f40ed2108677706b29a2250f185a7207a41b26939615efb"} err="failed to get container status \"9d5d3e4daab369ff5f40ed2108677706b29a2250f185a7207a41b26939615efb\": rpc error: code = NotFound desc = could not find container \"9d5d3e4daab369ff5f40ed2108677706b29a2250f185a7207a41b26939615efb\": container with ID starting with 9d5d3e4daab369ff5f40ed2108677706b29a2250f185a7207a41b26939615efb not found: ID does not exist" Nov 03 21:46:22.996651 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.996609 6524 scope.go:122] "RemoveContainer" containerID="50fd1fee4f8ba5e31f1b78656c421ef48ede3afd8e4f85f7fe26540d536d502f" Nov 03 21:46:22.996807 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.996793 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50fd1fee4f8ba5e31f1b78656c421ef48ede3afd8e4f85f7fe26540d536d502f"} err="failed to get container status \"50fd1fee4f8ba5e31f1b78656c421ef48ede3afd8e4f85f7fe26540d536d502f\": rpc error: code = NotFound desc = could not find container \"50fd1fee4f8ba5e31f1b78656c421ef48ede3afd8e4f85f7fe26540d536d502f\": container with ID starting with 50fd1fee4f8ba5e31f1b78656c421ef48ede3afd8e4f85f7fe26540d536d502f not found: ID does not exist" Nov 03 21:46:22.996858 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.996807 6524 scope.go:122] "RemoveContainer" containerID="64d36f5b414bd0de01d0058f60c37b3051fe2f67b3b5834c066ae2d742c4d487" Nov 03 21:46:22.996997 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.996982 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64d36f5b414bd0de01d0058f60c37b3051fe2f67b3b5834c066ae2d742c4d487"} err="failed to get container status \"64d36f5b414bd0de01d0058f60c37b3051fe2f67b3b5834c066ae2d742c4d487\": rpc error: code = NotFound desc = could not find container \"64d36f5b414bd0de01d0058f60c37b3051fe2f67b3b5834c066ae2d742c4d487\": container with ID starting with 64d36f5b414bd0de01d0058f60c37b3051fe2f67b3b5834c066ae2d742c4d487 not found: ID does not exist" Nov 03 21:46:22.997042 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.996998 6524 scope.go:122] "RemoveContainer" containerID="8fc4b9b4b217355d6d2c046bd20b54b62a6fa33031b4812bbe555ec7d6771f8d" Nov 03 21:46:22.997215 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.997198 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fc4b9b4b217355d6d2c046bd20b54b62a6fa33031b4812bbe555ec7d6771f8d"} err="failed to get container status \"8fc4b9b4b217355d6d2c046bd20b54b62a6fa33031b4812bbe555ec7d6771f8d\": rpc error: code = NotFound desc = could not find container \"8fc4b9b4b217355d6d2c046bd20b54b62a6fa33031b4812bbe555ec7d6771f8d\": container with ID starting with 8fc4b9b4b217355d6d2c046bd20b54b62a6fa33031b4812bbe555ec7d6771f8d not found: ID does not exist" Nov 03 21:46:22.997259 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.997216 6524 scope.go:122] "RemoveContainer" containerID="1f9121440b907bd547107b875553716b36cf846bf8b11f652a22435d05b362ab" Nov 03 21:46:22.997430 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.997412 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f9121440b907bd547107b875553716b36cf846bf8b11f652a22435d05b362ab"} err="failed to get container status \"1f9121440b907bd547107b875553716b36cf846bf8b11f652a22435d05b362ab\": rpc error: code = NotFound desc = could not find container \"1f9121440b907bd547107b875553716b36cf846bf8b11f652a22435d05b362ab\": container with ID starting with 1f9121440b907bd547107b875553716b36cf846bf8b11f652a22435d05b362ab not found: ID does not exist" Nov 03 21:46:22.997473 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.997430 6524 scope.go:122] "RemoveContainer" containerID="14b9495cfb235a428a51ba26e6ccea26a7e4b5c67bbe3b4f43a6d924eff2decf" Nov 03 21:46:22.997629 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.997612 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14b9495cfb235a428a51ba26e6ccea26a7e4b5c67bbe3b4f43a6d924eff2decf"} err="failed to get container status \"14b9495cfb235a428a51ba26e6ccea26a7e4b5c67bbe3b4f43a6d924eff2decf\": rpc error: code = NotFound desc = could not find container \"14b9495cfb235a428a51ba26e6ccea26a7e4b5c67bbe3b4f43a6d924eff2decf\": container with ID starting with 14b9495cfb235a428a51ba26e6ccea26a7e4b5c67bbe3b4f43a6d924eff2decf not found: ID does not exist" Nov 03 21:46:22.997673 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.997629 6524 scope.go:122] "RemoveContainer" containerID="a049ac28ba7cf6a3af10022353c8a212963dfa04ff032c77d3a428dac1c83f0e" Nov 03 21:46:22.997839 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.997823 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a049ac28ba7cf6a3af10022353c8a212963dfa04ff032c77d3a428dac1c83f0e"} err="failed to get container status \"a049ac28ba7cf6a3af10022353c8a212963dfa04ff032c77d3a428dac1c83f0e\": rpc error: code = NotFound desc = could not find container \"a049ac28ba7cf6a3af10022353c8a212963dfa04ff032c77d3a428dac1c83f0e\": container with ID starting with a049ac28ba7cf6a3af10022353c8a212963dfa04ff032c77d3a428dac1c83f0e not found: ID does not exist" Nov 03 21:46:22.997878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.997839 6524 scope.go:122] "RemoveContainer" containerID="9d5d3e4daab369ff5f40ed2108677706b29a2250f185a7207a41b26939615efb" Nov 03 21:46:22.998038 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.998023 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d5d3e4daab369ff5f40ed2108677706b29a2250f185a7207a41b26939615efb"} err="failed to get container status \"9d5d3e4daab369ff5f40ed2108677706b29a2250f185a7207a41b26939615efb\": rpc error: code = NotFound desc = could not find container \"9d5d3e4daab369ff5f40ed2108677706b29a2250f185a7207a41b26939615efb\": container with ID starting with 9d5d3e4daab369ff5f40ed2108677706b29a2250f185a7207a41b26939615efb not found: ID does not exist" Nov 03 21:46:22.998073 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.998038 6524 scope.go:122] "RemoveContainer" containerID="50fd1fee4f8ba5e31f1b78656c421ef48ede3afd8e4f85f7fe26540d536d502f" Nov 03 21:46:22.998255 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.998240 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50fd1fee4f8ba5e31f1b78656c421ef48ede3afd8e4f85f7fe26540d536d502f"} err="failed to get container status \"50fd1fee4f8ba5e31f1b78656c421ef48ede3afd8e4f85f7fe26540d536d502f\": rpc error: code = NotFound desc = could not find container \"50fd1fee4f8ba5e31f1b78656c421ef48ede3afd8e4f85f7fe26540d536d502f\": container with ID starting with 50fd1fee4f8ba5e31f1b78656c421ef48ede3afd8e4f85f7fe26540d536d502f not found: ID does not exist" Nov 03 21:46:22.998312 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.998255 6524 scope.go:122] "RemoveContainer" containerID="64d36f5b414bd0de01d0058f60c37b3051fe2f67b3b5834c066ae2d742c4d487" Nov 03 21:46:22.998487 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.998469 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64d36f5b414bd0de01d0058f60c37b3051fe2f67b3b5834c066ae2d742c4d487"} err="failed to get container status \"64d36f5b414bd0de01d0058f60c37b3051fe2f67b3b5834c066ae2d742c4d487\": rpc error: code = NotFound desc = could not find container \"64d36f5b414bd0de01d0058f60c37b3051fe2f67b3b5834c066ae2d742c4d487\": container with ID starting with 64d36f5b414bd0de01d0058f60c37b3051fe2f67b3b5834c066ae2d742c4d487 not found: ID does not exist" Nov 03 21:46:22.998487 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.998485 6524 scope.go:122] "RemoveContainer" containerID="8fc4b9b4b217355d6d2c046bd20b54b62a6fa33031b4812bbe555ec7d6771f8d" Nov 03 21:46:22.998719 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.998695 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fc4b9b4b217355d6d2c046bd20b54b62a6fa33031b4812bbe555ec7d6771f8d"} err="failed to get container status \"8fc4b9b4b217355d6d2c046bd20b54b62a6fa33031b4812bbe555ec7d6771f8d\": rpc error: code = NotFound desc = could not find container \"8fc4b9b4b217355d6d2c046bd20b54b62a6fa33031b4812bbe555ec7d6771f8d\": container with ID starting with 8fc4b9b4b217355d6d2c046bd20b54b62a6fa33031b4812bbe555ec7d6771f8d not found: ID does not exist" Nov 03 21:46:22.998719 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.998717 6524 scope.go:122] "RemoveContainer" containerID="1f9121440b907bd547107b875553716b36cf846bf8b11f652a22435d05b362ab" Nov 03 21:46:22.999049 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.999026 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f9121440b907bd547107b875553716b36cf846bf8b11f652a22435d05b362ab"} err="failed to get container status \"1f9121440b907bd547107b875553716b36cf846bf8b11f652a22435d05b362ab\": rpc error: code = NotFound desc = could not find container \"1f9121440b907bd547107b875553716b36cf846bf8b11f652a22435d05b362ab\": container with ID starting with 1f9121440b907bd547107b875553716b36cf846bf8b11f652a22435d05b362ab not found: ID does not exist" Nov 03 21:46:22.999049 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.999048 6524 scope.go:122] "RemoveContainer" containerID="14b9495cfb235a428a51ba26e6ccea26a7e4b5c67bbe3b4f43a6d924eff2decf" Nov 03 21:46:22.999428 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.999396 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14b9495cfb235a428a51ba26e6ccea26a7e4b5c67bbe3b4f43a6d924eff2decf"} err="failed to get container status \"14b9495cfb235a428a51ba26e6ccea26a7e4b5c67bbe3b4f43a6d924eff2decf\": rpc error: code = NotFound desc = could not find container \"14b9495cfb235a428a51ba26e6ccea26a7e4b5c67bbe3b4f43a6d924eff2decf\": container with ID starting with 14b9495cfb235a428a51ba26e6ccea26a7e4b5c67bbe3b4f43a6d924eff2decf not found: ID does not exist" Nov 03 21:46:22.999428 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.999427 6524 scope.go:122] "RemoveContainer" containerID="a049ac28ba7cf6a3af10022353c8a212963dfa04ff032c77d3a428dac1c83f0e" Nov 03 21:46:22.999683 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.999656 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a049ac28ba7cf6a3af10022353c8a212963dfa04ff032c77d3a428dac1c83f0e"} err="failed to get container status \"a049ac28ba7cf6a3af10022353c8a212963dfa04ff032c77d3a428dac1c83f0e\": rpc error: code = NotFound desc = could not find container \"a049ac28ba7cf6a3af10022353c8a212963dfa04ff032c77d3a428dac1c83f0e\": container with ID starting with a049ac28ba7cf6a3af10022353c8a212963dfa04ff032c77d3a428dac1c83f0e not found: ID does not exist" Nov 03 21:46:22.999683 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.999683 6524 scope.go:122] "RemoveContainer" containerID="9d5d3e4daab369ff5f40ed2108677706b29a2250f185a7207a41b26939615efb" Nov 03 21:46:22.999968 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:22.999727 6524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 03 21:46:23.000063 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000047 6524 cpu_manager.go:414] "containerMap: removing container" podUID="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" containerName="kube-rbac-proxy" Nov 03 21:46:23.000107 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000065 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" containerName="kube-rbac-proxy" Nov 03 21:46:23.000107 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000077 6524 cpu_manager.go:414] "containerMap: removing container" podUID="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" containerName="thanos-sidecar" Nov 03 21:46:23.000107 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000085 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" containerName="thanos-sidecar" Nov 03 21:46:23.000107 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000084 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d5d3e4daab369ff5f40ed2108677706b29a2250f185a7207a41b26939615efb"} err="failed to get container status \"9d5d3e4daab369ff5f40ed2108677706b29a2250f185a7207a41b26939615efb\": rpc error: code = NotFound desc = could not find container \"9d5d3e4daab369ff5f40ed2108677706b29a2250f185a7207a41b26939615efb\": container with ID starting with 9d5d3e4daab369ff5f40ed2108677706b29a2250f185a7207a41b26939615efb not found: ID does not exist" Nov 03 21:46:23.000107 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000096 6524 cpu_manager.go:414] "containerMap: removing container" podUID="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" containerName="alertmanager" Nov 03 21:46:23.000107 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000101 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" containerName="alertmanager" Nov 03 21:46:23.000319 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000113 6524 cpu_manager.go:414] "containerMap: removing container" podUID="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" containerName="init-config-reloader" Nov 03 21:46:23.000319 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000117 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" containerName="init-config-reloader" Nov 03 21:46:23.000319 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000143 6524 cpu_manager.go:414] "containerMap: removing container" podUID="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" containerName="prom-label-proxy" Nov 03 21:46:23.000319 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000151 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" containerName="prom-label-proxy" Nov 03 21:46:23.000319 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000160 6524 cpu_manager.go:414] "containerMap: removing container" podUID="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" containerName="kube-rbac-proxy" Nov 03 21:46:23.000319 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000166 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" containerName="kube-rbac-proxy" Nov 03 21:46:23.000319 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000174 6524 cpu_manager.go:414] "containerMap: removing container" podUID="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" containerName="kube-rbac-proxy-metric" Nov 03 21:46:23.000319 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000178 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" containerName="kube-rbac-proxy-metric" Nov 03 21:46:23.000319 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000188 6524 cpu_manager.go:414] "containerMap: removing container" podUID="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" containerName="config-reloader" Nov 03 21:46:23.000319 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000192 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" containerName="config-reloader" Nov 03 21:46:23.000319 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000100 6524 scope.go:122] "RemoveContainer" containerID="50fd1fee4f8ba5e31f1b78656c421ef48ede3afd8e4f85f7fe26540d536d502f" Nov 03 21:46:23.000319 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000198 6524 cpu_manager.go:414] "containerMap: removing container" podUID="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" containerName="prometheus" Nov 03 21:46:23.000319 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000202 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" containerName="prometheus" Nov 03 21:46:23.000319 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000211 6524 cpu_manager.go:414] "containerMap: removing container" podUID="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" containerName="kube-rbac-proxy-thanos" Nov 03 21:46:23.000319 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000218 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" containerName="kube-rbac-proxy-thanos" Nov 03 21:46:23.000319 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000229 6524 cpu_manager.go:414] "containerMap: removing container" podUID="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" containerName="config-reloader" Nov 03 21:46:23.000319 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000235 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" containerName="config-reloader" Nov 03 21:46:23.000319 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000244 6524 cpu_manager.go:414] "containerMap: removing container" podUID="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" containerName="init-config-reloader" Nov 03 21:46:23.000319 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000248 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" containerName="init-config-reloader" Nov 03 21:46:23.000319 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000257 6524 cpu_manager.go:414] "containerMap: removing container" podUID="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" containerName="kube-rbac-proxy-web" Nov 03 21:46:23.000319 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000261 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" containerName="kube-rbac-proxy-web" Nov 03 21:46:23.000319 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000267 6524 cpu_manager.go:414] "containerMap: removing container" podUID="866ddb5a-c929-40c1-911d-ca40c5405723" containerName="console" Nov 03 21:46:23.000319 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000271 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="866ddb5a-c929-40c1-911d-ca40c5405723" containerName="console" Nov 03 21:46:23.000319 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000279 6524 cpu_manager.go:414] "containerMap: removing container" podUID="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" containerName="kube-rbac-proxy-web" Nov 03 21:46:23.000319 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000284 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" containerName="kube-rbac-proxy-web" Nov 03 21:46:23.001098 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000341 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" containerName="init-config-reloader" Nov 03 21:46:23.001098 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000353 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" containerName="prom-label-proxy" Nov 03 21:46:23.001098 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000362 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" containerName="kube-rbac-proxy" Nov 03 21:46:23.001098 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000369 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" containerName="kube-rbac-proxy-metric" Nov 03 21:46:23.001098 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000379 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" containerName="config-reloader" Nov 03 21:46:23.001098 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000387 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" containerName="prometheus" Nov 03 21:46:23.001098 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000392 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" containerName="kube-rbac-proxy-thanos" Nov 03 21:46:23.001098 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000396 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" containerName="config-reloader" Nov 03 21:46:23.001098 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000401 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" containerName="init-config-reloader" Nov 03 21:46:23.001098 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000406 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" containerName="kube-rbac-proxy-web" Nov 03 21:46:23.001098 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000410 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="866ddb5a-c929-40c1-911d-ca40c5405723" containerName="console" Nov 03 21:46:23.001098 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000415 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" containerName="kube-rbac-proxy-web" Nov 03 21:46:23.001098 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000420 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" containerName="kube-rbac-proxy" Nov 03 21:46:23.001098 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000424 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" containerName="thanos-sidecar" Nov 03 21:46:23.001098 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000429 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="a2ccde5e-94a2-47ac-aff9-65248e4a6c92" containerName="alertmanager" Nov 03 21:46:23.001098 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000451 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50fd1fee4f8ba5e31f1b78656c421ef48ede3afd8e4f85f7fe26540d536d502f"} err="failed to get container status \"50fd1fee4f8ba5e31f1b78656c421ef48ede3afd8e4f85f7fe26540d536d502f\": rpc error: code = NotFound desc = could not find container \"50fd1fee4f8ba5e31f1b78656c421ef48ede3afd8e4f85f7fe26540d536d502f\": container with ID starting with 50fd1fee4f8ba5e31f1b78656c421ef48ede3afd8e4f85f7fe26540d536d502f not found: ID does not exist" Nov 03 21:46:23.001098 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000468 6524 scope.go:122] "RemoveContainer" containerID="64d36f5b414bd0de01d0058f60c37b3051fe2f67b3b5834c066ae2d742c4d487" Nov 03 21:46:23.001098 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000726 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64d36f5b414bd0de01d0058f60c37b3051fe2f67b3b5834c066ae2d742c4d487"} err="failed to get container status \"64d36f5b414bd0de01d0058f60c37b3051fe2f67b3b5834c066ae2d742c4d487\": rpc error: code = NotFound desc = could not find container \"64d36f5b414bd0de01d0058f60c37b3051fe2f67b3b5834c066ae2d742c4d487\": container with ID starting with 64d36f5b414bd0de01d0058f60c37b3051fe2f67b3b5834c066ae2d742c4d487 not found: ID does not exist" Nov 03 21:46:23.001098 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000743 6524 scope.go:122] "RemoveContainer" containerID="8fc4b9b4b217355d6d2c046bd20b54b62a6fa33031b4812bbe555ec7d6771f8d" Nov 03 21:46:23.001098 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000969 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fc4b9b4b217355d6d2c046bd20b54b62a6fa33031b4812bbe555ec7d6771f8d"} err="failed to get container status \"8fc4b9b4b217355d6d2c046bd20b54b62a6fa33031b4812bbe555ec7d6771f8d\": rpc error: code = NotFound desc = could not find container \"8fc4b9b4b217355d6d2c046bd20b54b62a6fa33031b4812bbe555ec7d6771f8d\": container with ID starting with 8fc4b9b4b217355d6d2c046bd20b54b62a6fa33031b4812bbe555ec7d6771f8d not found: ID does not exist" Nov 03 21:46:23.001098 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.000986 6524 scope.go:122] "RemoveContainer" containerID="1f9121440b907bd547107b875553716b36cf846bf8b11f652a22435d05b362ab" Nov 03 21:46:23.001791 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.001218 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f9121440b907bd547107b875553716b36cf846bf8b11f652a22435d05b362ab"} err="failed to get container status \"1f9121440b907bd547107b875553716b36cf846bf8b11f652a22435d05b362ab\": rpc error: code = NotFound desc = could not find container \"1f9121440b907bd547107b875553716b36cf846bf8b11f652a22435d05b362ab\": container with ID starting with 1f9121440b907bd547107b875553716b36cf846bf8b11f652a22435d05b362ab not found: ID does not exist" Nov 03 21:46:23.001791 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.001238 6524 scope.go:122] "RemoveContainer" containerID="14b9495cfb235a428a51ba26e6ccea26a7e4b5c67bbe3b4f43a6d924eff2decf" Nov 03 21:46:23.001791 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.001444 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14b9495cfb235a428a51ba26e6ccea26a7e4b5c67bbe3b4f43a6d924eff2decf"} err="failed to get container status \"14b9495cfb235a428a51ba26e6ccea26a7e4b5c67bbe3b4f43a6d924eff2decf\": rpc error: code = NotFound desc = could not find container \"14b9495cfb235a428a51ba26e6ccea26a7e4b5c67bbe3b4f43a6d924eff2decf\": container with ID starting with 14b9495cfb235a428a51ba26e6ccea26a7e4b5c67bbe3b4f43a6d924eff2decf not found: ID does not exist" Nov 03 21:46:23.001791 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.001474 6524 scope.go:122] "RemoveContainer" containerID="a049ac28ba7cf6a3af10022353c8a212963dfa04ff032c77d3a428dac1c83f0e" Nov 03 21:46:23.001791 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.001700 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a049ac28ba7cf6a3af10022353c8a212963dfa04ff032c77d3a428dac1c83f0e"} err="failed to get container status \"a049ac28ba7cf6a3af10022353c8a212963dfa04ff032c77d3a428dac1c83f0e\": rpc error: code = NotFound desc = could not find container \"a049ac28ba7cf6a3af10022353c8a212963dfa04ff032c77d3a428dac1c83f0e\": container with ID starting with a049ac28ba7cf6a3af10022353c8a212963dfa04ff032c77d3a428dac1c83f0e not found: ID does not exist" Nov 03 21:46:23.001791 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.001717 6524 scope.go:122] "RemoveContainer" containerID="9d5d3e4daab369ff5f40ed2108677706b29a2250f185a7207a41b26939615efb" Nov 03 21:46:23.001991 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.001920 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d5d3e4daab369ff5f40ed2108677706b29a2250f185a7207a41b26939615efb"} err="failed to get container status \"9d5d3e4daab369ff5f40ed2108677706b29a2250f185a7207a41b26939615efb\": rpc error: code = NotFound desc = could not find container \"9d5d3e4daab369ff5f40ed2108677706b29a2250f185a7207a41b26939615efb\": container with ID starting with 9d5d3e4daab369ff5f40ed2108677706b29a2250f185a7207a41b26939615efb not found: ID does not exist" Nov 03 21:46:23.001991 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.001946 6524 scope.go:122] "RemoveContainer" containerID="50fd1fee4f8ba5e31f1b78656c421ef48ede3afd8e4f85f7fe26540d536d502f" Nov 03 21:46:23.002200 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.002181 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50fd1fee4f8ba5e31f1b78656c421ef48ede3afd8e4f85f7fe26540d536d502f"} err="failed to get container status \"50fd1fee4f8ba5e31f1b78656c421ef48ede3afd8e4f85f7fe26540d536d502f\": rpc error: code = NotFound desc = could not find container \"50fd1fee4f8ba5e31f1b78656c421ef48ede3afd8e4f85f7fe26540d536d502f\": container with ID starting with 50fd1fee4f8ba5e31f1b78656c421ef48ede3afd8e4f85f7fe26540d536d502f not found: ID does not exist" Nov 03 21:46:23.002309 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.002201 6524 scope.go:122] "RemoveContainer" containerID="64d36f5b414bd0de01d0058f60c37b3051fe2f67b3b5834c066ae2d742c4d487" Nov 03 21:46:23.002448 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.002429 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64d36f5b414bd0de01d0058f60c37b3051fe2f67b3b5834c066ae2d742c4d487"} err="failed to get container status \"64d36f5b414bd0de01d0058f60c37b3051fe2f67b3b5834c066ae2d742c4d487\": rpc error: code = NotFound desc = could not find container \"64d36f5b414bd0de01d0058f60c37b3051fe2f67b3b5834c066ae2d742c4d487\": container with ID starting with 64d36f5b414bd0de01d0058f60c37b3051fe2f67b3b5834c066ae2d742c4d487 not found: ID does not exist" Nov 03 21:46:23.002538 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.002449 6524 scope.go:122] "RemoveContainer" containerID="8fc4b9b4b217355d6d2c046bd20b54b62a6fa33031b4812bbe555ec7d6771f8d" Nov 03 21:46:23.002696 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.002677 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fc4b9b4b217355d6d2c046bd20b54b62a6fa33031b4812bbe555ec7d6771f8d"} err="failed to get container status \"8fc4b9b4b217355d6d2c046bd20b54b62a6fa33031b4812bbe555ec7d6771f8d\": rpc error: code = NotFound desc = could not find container \"8fc4b9b4b217355d6d2c046bd20b54b62a6fa33031b4812bbe555ec7d6771f8d\": container with ID starting with 8fc4b9b4b217355d6d2c046bd20b54b62a6fa33031b4812bbe555ec7d6771f8d not found: ID does not exist" Nov 03 21:46:23.002749 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.002696 6524 scope.go:122] "RemoveContainer" containerID="1f9121440b907bd547107b875553716b36cf846bf8b11f652a22435d05b362ab" Nov 03 21:46:23.002906 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.002890 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f9121440b907bd547107b875553716b36cf846bf8b11f652a22435d05b362ab"} err="failed to get container status \"1f9121440b907bd547107b875553716b36cf846bf8b11f652a22435d05b362ab\": rpc error: code = NotFound desc = could not find container \"1f9121440b907bd547107b875553716b36cf846bf8b11f652a22435d05b362ab\": container with ID starting with 1f9121440b907bd547107b875553716b36cf846bf8b11f652a22435d05b362ab not found: ID does not exist" Nov 03 21:46:23.002949 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.002907 6524 scope.go:122] "RemoveContainer" containerID="14b9495cfb235a428a51ba26e6ccea26a7e4b5c67bbe3b4f43a6d924eff2decf" Nov 03 21:46:23.003115 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.003098 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14b9495cfb235a428a51ba26e6ccea26a7e4b5c67bbe3b4f43a6d924eff2decf"} err="failed to get container status \"14b9495cfb235a428a51ba26e6ccea26a7e4b5c67bbe3b4f43a6d924eff2decf\": rpc error: code = NotFound desc = could not find container \"14b9495cfb235a428a51ba26e6ccea26a7e4b5c67bbe3b4f43a6d924eff2decf\": container with ID starting with 14b9495cfb235a428a51ba26e6ccea26a7e4b5c67bbe3b4f43a6d924eff2decf not found: ID does not exist" Nov 03 21:46:23.004295 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.004279 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.011407 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.011321 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Nov 03 21:46:23.011407 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.011358 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Nov 03 21:46:23.011407 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.011359 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Nov 03 21:46:23.011663 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.011436 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Nov 03 21:46:23.011663 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.011587 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Nov 03 21:46:23.011663 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.011606 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Nov 03 21:46:23.011663 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.011629 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Nov 03 21:46:23.011663 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.011628 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Nov 03 21:46:23.011663 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.011646 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Nov 03 21:46:23.011663 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.011660 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Nov 03 21:46:23.011663 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.011648 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Nov 03 21:46:23.011663 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.011649 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Nov 03 21:46:23.012407 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.012393 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Nov 03 21:46:23.012960 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.012946 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-6hj4n\"" Nov 03 21:46:23.013003 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.012949 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-52l5qoid38f49\"" Nov 03 21:46:23.019996 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.019978 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Nov 03 21:46:23.020110 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.020095 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Nov 03 21:46:23.023449 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.023416 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 03 21:46:23.120878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.120841 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/327ba0ad-9702-4e8b-bd97-8b6792b03e51-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.120878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.120882 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/327ba0ad-9702-4e8b-bd97-8b6792b03e51-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.121089 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.120904 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/327ba0ad-9702-4e8b-bd97-8b6792b03e51-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.121089 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.120974 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/327ba0ad-9702-4e8b-bd97-8b6792b03e51-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.121089 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.121022 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9l7n\" (UniqueName: \"kubernetes.io/projected/327ba0ad-9702-4e8b-bd97-8b6792b03e51-kube-api-access-p9l7n\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.121089 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.121053 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/327ba0ad-9702-4e8b-bd97-8b6792b03e51-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.121089 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.121069 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/327ba0ad-9702-4e8b-bd97-8b6792b03e51-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.121089 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.121085 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/327ba0ad-9702-4e8b-bd97-8b6792b03e51-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.121294 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.121193 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/327ba0ad-9702-4e8b-bd97-8b6792b03e51-config-out\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.121294 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.121212 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/327ba0ad-9702-4e8b-bd97-8b6792b03e51-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.121294 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.121242 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/327ba0ad-9702-4e8b-bd97-8b6792b03e51-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.121381 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.121319 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/327ba0ad-9702-4e8b-bd97-8b6792b03e51-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.121381 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.121339 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/327ba0ad-9702-4e8b-bd97-8b6792b03e51-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.121381 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.121357 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/327ba0ad-9702-4e8b-bd97-8b6792b03e51-config\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.121469 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.121389 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/327ba0ad-9702-4e8b-bd97-8b6792b03e51-web-config\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.121469 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.121425 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/327ba0ad-9702-4e8b-bd97-8b6792b03e51-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.121469 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.121442 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/327ba0ad-9702-4e8b-bd97-8b6792b03e51-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.121552 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.121483 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/327ba0ad-9702-4e8b-bd97-8b6792b03e51-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.121552 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.121508 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/327ba0ad-9702-4e8b-bd97-8b6792b03e51-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.121552 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.121524 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/327ba0ad-9702-4e8b-bd97-8b6792b03e51-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.222481 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.222396 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/327ba0ad-9702-4e8b-bd97-8b6792b03e51-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.222481 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.222436 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/327ba0ad-9702-4e8b-bd97-8b6792b03e51-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.222481 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.222454 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/327ba0ad-9702-4e8b-bd97-8b6792b03e51-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.222481 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.222478 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/327ba0ad-9702-4e8b-bd97-8b6792b03e51-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.222828 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.222497 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/327ba0ad-9702-4e8b-bd97-8b6792b03e51-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.222828 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.222516 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/327ba0ad-9702-4e8b-bd97-8b6792b03e51-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.222828 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.222530 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/327ba0ad-9702-4e8b-bd97-8b6792b03e51-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.222828 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.222549 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9l7n\" (UniqueName: \"kubernetes.io/projected/327ba0ad-9702-4e8b-bd97-8b6792b03e51-kube-api-access-p9l7n\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.222828 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.222573 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/327ba0ad-9702-4e8b-bd97-8b6792b03e51-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.223460 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.222898 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/327ba0ad-9702-4e8b-bd97-8b6792b03e51-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.223460 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.222942 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/327ba0ad-9702-4e8b-bd97-8b6792b03e51-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.223460 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.223014 6524 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Nov 03 21:46:23.223460 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.223076 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/327ba0ad-9702-4e8b-bd97-8b6792b03e51-config-out\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.223460 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.223143 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/327ba0ad-9702-4e8b-bd97-8b6792b03e51-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.223460 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.223223 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/327ba0ad-9702-4e8b-bd97-8b6792b03e51-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.223460 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.223395 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/327ba0ad-9702-4e8b-bd97-8b6792b03e51-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.224199 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.223485 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/327ba0ad-9702-4e8b-bd97-8b6792b03e51-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.224199 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.223553 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/327ba0ad-9702-4e8b-bd97-8b6792b03e51-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.224199 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.223581 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/327ba0ad-9702-4e8b-bd97-8b6792b03e51-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.224199 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.223634 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/327ba0ad-9702-4e8b-bd97-8b6792b03e51-config\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.224199 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.223642 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/327ba0ad-9702-4e8b-bd97-8b6792b03e51-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.224199 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.223636 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/327ba0ad-9702-4e8b-bd97-8b6792b03e51-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.224199 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.223676 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/327ba0ad-9702-4e8b-bd97-8b6792b03e51-web-config\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.224199 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.223683 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/327ba0ad-9702-4e8b-bd97-8b6792b03e51-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.224199 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.223733 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/327ba0ad-9702-4e8b-bd97-8b6792b03e51-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.224199 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.223766 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/327ba0ad-9702-4e8b-bd97-8b6792b03e51-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.225163 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.224722 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/327ba0ad-9702-4e8b-bd97-8b6792b03e51-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.225248 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.225225 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/327ba0ad-9702-4e8b-bd97-8b6792b03e51-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.226091 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.226063 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/327ba0ad-9702-4e8b-bd97-8b6792b03e51-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.226695 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.226526 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/327ba0ad-9702-4e8b-bd97-8b6792b03e51-web-config\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.226695 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.226635 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/327ba0ad-9702-4e8b-bd97-8b6792b03e51-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.226846 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.226781 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/327ba0ad-9702-4e8b-bd97-8b6792b03e51-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.227514 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.227486 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/327ba0ad-9702-4e8b-bd97-8b6792b03e51-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.227655 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.227626 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/327ba0ad-9702-4e8b-bd97-8b6792b03e51-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.227808 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.227702 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/327ba0ad-9702-4e8b-bd97-8b6792b03e51-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.227897 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.227860 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/327ba0ad-9702-4e8b-bd97-8b6792b03e51-config-out\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.228242 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.228222 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/327ba0ad-9702-4e8b-bd97-8b6792b03e51-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.228365 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.228342 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/327ba0ad-9702-4e8b-bd97-8b6792b03e51-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.228713 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.228692 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/327ba0ad-9702-4e8b-bd97-8b6792b03e51-config\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.228969 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.228953 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/327ba0ad-9702-4e8b-bd97-8b6792b03e51-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.235528 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.235509 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9l7n\" (UniqueName: \"kubernetes.io/projected/327ba0ad-9702-4e8b-bd97-8b6792b03e51-kube-api-access-p9l7n\") pod \"prometheus-k8s-0\" (UID: \"327ba0ad-9702-4e8b-bd97-8b6792b03e51\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.320624 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.320587 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:23.517794 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.517719 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x26mj" Nov 03 21:46:23.609762 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.609727 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 03 21:46:23.613768 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:46:23.613735 6524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod327ba0ad_9702_4e8b_bd97_8b6792b03e51.slice/crio-d4c70d99d3bec706960c5451d5c2d8d2195dcb15061ee16a064d074041298ca2 WatchSource:0}: Error finding container d4c70d99d3bec706960c5451d5c2d8d2195dcb15061ee16a064d074041298ca2: Status 404 returned error can't find the container with id d4c70d99d3bec706960c5451d5c2d8d2195dcb15061ee16a064d074041298ca2 Nov 03 21:46:23.946708 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.946663 6524 generic.go:356] "Generic (PLEG): container finished" podID="327ba0ad-9702-4e8b-bd97-8b6792b03e51" containerID="e5ec8a40170c13140620e0dd09b1cb10ae22661f81f0bd348ee76c90463fff53" exitCode=0 Nov 03 21:46:23.947112 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.946742 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"327ba0ad-9702-4e8b-bd97-8b6792b03e51","Type":"ContainerDied","Data":"e5ec8a40170c13140620e0dd09b1cb10ae22661f81f0bd348ee76c90463fff53"} Nov 03 21:46:23.947112 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:23.946855 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"327ba0ad-9702-4e8b-bd97-8b6792b03e51","Type":"ContainerStarted","Data":"d4c70d99d3bec706960c5451d5c2d8d2195dcb15061ee16a064d074041298ca2"} Nov 03 21:46:24.773523 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:24.773487 6524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1a5cbb44-3dbf-40a1-916b-71465d2dc2ea" path="/var/lib/kubelet/pods/1a5cbb44-3dbf-40a1-916b-71465d2dc2ea/volumes" Nov 03 21:46:24.955500 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:24.955461 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"327ba0ad-9702-4e8b-bd97-8b6792b03e51","Type":"ContainerStarted","Data":"ff06c3001ccf2ad501db9557be562f5fe473c79216360ebe8471c7d96dac70c7"} Nov 03 21:46:24.955875 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:24.955537 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"327ba0ad-9702-4e8b-bd97-8b6792b03e51","Type":"ContainerStarted","Data":"4968140b80988721f052732f7d59dd9ef60bb9a11e1ca881f8df17625ca0ac97"} Nov 03 21:46:24.955875 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:24.955566 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"327ba0ad-9702-4e8b-bd97-8b6792b03e51","Type":"ContainerStarted","Data":"64250e8187997e35f30542f1298ef002bc9cb81dc47d7fe29f1c627aa20978b3"} Nov 03 21:46:24.955875 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:24.955594 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"327ba0ad-9702-4e8b-bd97-8b6792b03e51","Type":"ContainerStarted","Data":"8208813b1cedef811fe5d132121ea6b1c73ac0d2fa9a2a44c34c9e14e607b759"} Nov 03 21:46:24.955875 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:24.955621 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"327ba0ad-9702-4e8b-bd97-8b6792b03e51","Type":"ContainerStarted","Data":"6c0cf883e69e55600d5cc14902dd289370298ad6e1325b39119b8b0a3f0f993c"} Nov 03 21:46:24.955875 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:24.955644 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"327ba0ad-9702-4e8b-bd97-8b6792b03e51","Type":"ContainerStarted","Data":"26a14249bf4bb8f8fe68fc681a888e5b4eabdbf9d806e5cf1dc947e96467081d"} Nov 03 21:46:24.994401 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:24.994331 6524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.978033346 podStartE2EDuration="2.994314914s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.281568ms" isStatefulPod=false podCreationTimestamp="2026-11-03 21:46:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:46:24.992668327 +0000 UTC m=+30.344995131" watchObservedRunningTime="2026-11-03 21:46:24.994314914 +0000 UTC m=+30.346641720" Nov 03 21:46:28.320859 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:28.320817 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:29.706689 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:29.706644 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 21:46:30.344880 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:30.344850 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-zrbsd" Nov 03 21:46:34.629310 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:34.629271 6524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:34.645044 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:34.645017 6524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:35.007304 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:46:35.007232 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Nov 03 21:46:59.706839 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:46:59.706798 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 21:47:29.707513 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:47:29.707469 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 21:47:59.706921 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:47:59.706829 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 21:48:29.705307 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:48:29.705272 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 21:48:59.707087 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:48:59.707042 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 21:49:20.268775 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:20.268698 6524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-wkf85"] Nov 03 21:49:20.271915 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:20.271895 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-wkf85" Nov 03 21:49:20.281601 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:20.281574 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Nov 03 21:49:20.281722 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:20.281708 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Nov 03 21:49:20.281946 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:20.281926 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-wkf85"] Nov 03 21:49:20.282872 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:20.282856 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-9knbs\"" Nov 03 21:49:20.319342 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:20.319301 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gb6rp\" (UniqueName: \"kubernetes.io/projected/fa6d73f4-ed1a-46b8-94d5-7637b791fa89-kube-api-access-gb6rp\") pod \"cert-manager-cainjector-5c57bbd4fc-wkf85\" (UID: \"fa6d73f4-ed1a-46b8-94d5-7637b791fa89\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-wkf85" Nov 03 21:49:20.319515 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:20.319376 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fa6d73f4-ed1a-46b8-94d5-7637b791fa89-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-wkf85\" (UID: \"fa6d73f4-ed1a-46b8-94d5-7637b791fa89\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-wkf85" Nov 03 21:49:20.420073 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:20.420017 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fa6d73f4-ed1a-46b8-94d5-7637b791fa89-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-wkf85\" (UID: \"fa6d73f4-ed1a-46b8-94d5-7637b791fa89\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-wkf85" Nov 03 21:49:20.420282 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:20.420091 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gb6rp\" (UniqueName: \"kubernetes.io/projected/fa6d73f4-ed1a-46b8-94d5-7637b791fa89-kube-api-access-gb6rp\") pod \"cert-manager-cainjector-5c57bbd4fc-wkf85\" (UID: \"fa6d73f4-ed1a-46b8-94d5-7637b791fa89\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-wkf85" Nov 03 21:49:20.433335 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:20.433302 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fa6d73f4-ed1a-46b8-94d5-7637b791fa89-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-wkf85\" (UID: \"fa6d73f4-ed1a-46b8-94d5-7637b791fa89\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-wkf85" Nov 03 21:49:20.433448 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:20.433375 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gb6rp\" (UniqueName: \"kubernetes.io/projected/fa6d73f4-ed1a-46b8-94d5-7637b791fa89-kube-api-access-gb6rp\") pod \"cert-manager-cainjector-5c57bbd4fc-wkf85\" (UID: \"fa6d73f4-ed1a-46b8-94d5-7637b791fa89\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-wkf85" Nov 03 21:49:20.598703 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:20.598670 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-wkf85" Nov 03 21:49:20.868991 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:20.868917 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-wkf85"] Nov 03 21:49:20.872817 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:49:20.872791 6524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfa6d73f4_ed1a_46b8_94d5_7637b791fa89.slice/crio-068e31c5784836f5f4872f709c383cbef657803707f85d90de2cbae2cd8695d3 WatchSource:0}: Error finding container 068e31c5784836f5f4872f709c383cbef657803707f85d90de2cbae2cd8695d3: Status 404 returned error can't find the container with id 068e31c5784836f5f4872f709c383cbef657803707f85d90de2cbae2cd8695d3 Nov 03 21:49:20.874806 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:20.874789 6524 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 03 21:49:21.528372 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:21.528335 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-wkf85" event={"ID":"fa6d73f4-ed1a-46b8-94d5-7637b791fa89","Type":"ContainerStarted","Data":"068e31c5784836f5f4872f709c383cbef657803707f85d90de2cbae2cd8695d3"} Nov 03 21:49:24.542679 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:24.542588 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-wkf85" event={"ID":"fa6d73f4-ed1a-46b8-94d5-7637b791fa89","Type":"ContainerStarted","Data":"9f4fd8e85d7a64910e4f4ab83b0219737c99315afe4ad6c970edc8dafd0fc178"} Nov 03 21:49:24.560787 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:24.560724 6524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-wkf85" podStartSLOduration=1.2462102019999999 podStartE2EDuration="4.560707233s" totalImagesPullingTime="3.314497031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:49:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:49:24.560635771 +0000 UTC m=+209.912962575" watchObservedRunningTime="2026-11-03 21:49:24.560707233 +0000 UTC m=+209.913034036" Nov 03 21:49:29.707444 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:49:29.707400 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 21:49:36.482858 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:36.482823 6524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-8465994787-6kp69"] Nov 03 21:49:36.491852 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:36.491826 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-8465994787-6kp69" Nov 03 21:49:36.495546 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:36.495518 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-8465994787-6kp69"] Nov 03 21:49:36.496697 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:36.496671 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Nov 03 21:49:36.496826 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:36.496711 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Nov 03 21:49:36.496826 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:36.496758 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-cfjt9\"" Nov 03 21:49:36.553460 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:36.553382 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prhqw\" (UniqueName: \"kubernetes.io/projected/5806ca3d-887b-4b37-a0fe-31303425d395-kube-api-access-prhqw\") pod \"openshift-lws-operator-8465994787-6kp69\" (UID: \"5806ca3d-887b-4b37-a0fe-31303425d395\") " pod="openshift-lws-operator/openshift-lws-operator-8465994787-6kp69" Nov 03 21:49:36.553606 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:36.553504 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5806ca3d-887b-4b37-a0fe-31303425d395-tmp\") pod \"openshift-lws-operator-8465994787-6kp69\" (UID: \"5806ca3d-887b-4b37-a0fe-31303425d395\") " pod="openshift-lws-operator/openshift-lws-operator-8465994787-6kp69" Nov 03 21:49:36.654274 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:36.654233 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5806ca3d-887b-4b37-a0fe-31303425d395-tmp\") pod \"openshift-lws-operator-8465994787-6kp69\" (UID: \"5806ca3d-887b-4b37-a0fe-31303425d395\") " pod="openshift-lws-operator/openshift-lws-operator-8465994787-6kp69" Nov 03 21:49:36.654274 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:36.654278 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prhqw\" (UniqueName: \"kubernetes.io/projected/5806ca3d-887b-4b37-a0fe-31303425d395-kube-api-access-prhqw\") pod \"openshift-lws-operator-8465994787-6kp69\" (UID: \"5806ca3d-887b-4b37-a0fe-31303425d395\") " pod="openshift-lws-operator/openshift-lws-operator-8465994787-6kp69" Nov 03 21:49:36.654701 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:36.654679 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5806ca3d-887b-4b37-a0fe-31303425d395-tmp\") pod \"openshift-lws-operator-8465994787-6kp69\" (UID: \"5806ca3d-887b-4b37-a0fe-31303425d395\") " pod="openshift-lws-operator/openshift-lws-operator-8465994787-6kp69" Nov 03 21:49:36.666864 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:36.666832 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-prhqw\" (UniqueName: \"kubernetes.io/projected/5806ca3d-887b-4b37-a0fe-31303425d395-kube-api-access-prhqw\") pod \"openshift-lws-operator-8465994787-6kp69\" (UID: \"5806ca3d-887b-4b37-a0fe-31303425d395\") " pod="openshift-lws-operator/openshift-lws-operator-8465994787-6kp69" Nov 03 21:49:36.813183 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:36.813078 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-8465994787-6kp69" Nov 03 21:49:37.093954 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:37.093922 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-8465994787-6kp69"] Nov 03 21:49:37.097674 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:49:37.097646 6524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5806ca3d_887b_4b37_a0fe_31303425d395.slice/crio-d44d3e1f7221e6af9dece2fb2d776f2208b454614c2d01849783c7e475c8e20f WatchSource:0}: Error finding container d44d3e1f7221e6af9dece2fb2d776f2208b454614c2d01849783c7e475c8e20f: Status 404 returned error can't find the container with id d44d3e1f7221e6af9dece2fb2d776f2208b454614c2d01849783c7e475c8e20f Nov 03 21:49:37.587485 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:37.587445 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-8465994787-6kp69" event={"ID":"5806ca3d-887b-4b37-a0fe-31303425d395","Type":"ContainerStarted","Data":"d44d3e1f7221e6af9dece2fb2d776f2208b454614c2d01849783c7e475c8e20f"} Nov 03 21:49:40.601098 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:40.601063 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-8465994787-6kp69" event={"ID":"5806ca3d-887b-4b37-a0fe-31303425d395","Type":"ContainerStarted","Data":"3ef88e3841d5b55cb2a2a2ead033ad4d9018f24d67fb93ecc27ae9013963ce97"} Nov 03 21:49:40.620284 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:40.620231 6524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-8465994787-6kp69" podStartSLOduration=1.758322731 podStartE2EDuration="4.620217585s" totalImagesPullingTime="2.861894854s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:49:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:49:40.619368348 +0000 UTC m=+225.971695167" watchObservedRunningTime="2026-11-03 21:49:40.620217585 +0000 UTC m=+225.972544388" Nov 03 21:49:54.850903 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:54.850868 6524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-77f6458f94-jpmrh"] Nov 03 21:49:54.854280 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:54.854264 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-77f6458f94-jpmrh" Nov 03 21:49:54.883060 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:54.883030 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-77f6458f94-jpmrh"] Nov 03 21:49:54.921631 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:54.921580 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Nov 03 21:49:54.921893 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:54.921871 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Nov 03 21:49:54.928634 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:54.928608 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Nov 03 21:49:54.928737 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:54.928723 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Nov 03 21:49:54.928895 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:54.928882 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-metrics-tls\"" Nov 03 21:49:54.928934 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:54.928897 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-c5jhj\"" Nov 03 21:49:55.007693 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:55.007664 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0f1eb659-a77c-4638-8326-6be9dc29e08d-manifests\") pod \"opendatahub-operator-controller-manager-77f6458f94-jpmrh\" (UID: \"0f1eb659-a77c-4638-8326-6be9dc29e08d\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6458f94-jpmrh" Nov 03 21:49:55.007873 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:55.007699 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gltdm\" (UniqueName: \"kubernetes.io/projected/0f1eb659-a77c-4638-8326-6be9dc29e08d-kube-api-access-gltdm\") pod \"opendatahub-operator-controller-manager-77f6458f94-jpmrh\" (UID: \"0f1eb659-a77c-4638-8326-6be9dc29e08d\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6458f94-jpmrh" Nov 03 21:49:55.007873 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:55.007722 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0f1eb659-a77c-4638-8326-6be9dc29e08d-tmp\") pod \"opendatahub-operator-controller-manager-77f6458f94-jpmrh\" (UID: \"0f1eb659-a77c-4638-8326-6be9dc29e08d\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6458f94-jpmrh" Nov 03 21:49:55.007873 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:55.007794 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0f1eb659-a77c-4638-8326-6be9dc29e08d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-77f6458f94-jpmrh\" (UID: \"0f1eb659-a77c-4638-8326-6be9dc29e08d\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6458f94-jpmrh" Nov 03 21:49:55.007873 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:55.007854 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0f1eb659-a77c-4638-8326-6be9dc29e08d-webhook-cert\") pod \"opendatahub-operator-controller-manager-77f6458f94-jpmrh\" (UID: \"0f1eb659-a77c-4638-8326-6be9dc29e08d\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6458f94-jpmrh" Nov 03 21:49:55.008031 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:55.007882 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0f1eb659-a77c-4638-8326-6be9dc29e08d-metrics-cert\") pod \"opendatahub-operator-controller-manager-77f6458f94-jpmrh\" (UID: \"0f1eb659-a77c-4638-8326-6be9dc29e08d\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6458f94-jpmrh" Nov 03 21:49:55.108990 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:55.108897 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0f1eb659-a77c-4638-8326-6be9dc29e08d-webhook-cert\") pod \"opendatahub-operator-controller-manager-77f6458f94-jpmrh\" (UID: \"0f1eb659-a77c-4638-8326-6be9dc29e08d\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6458f94-jpmrh" Nov 03 21:49:55.108990 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:55.108944 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0f1eb659-a77c-4638-8326-6be9dc29e08d-metrics-cert\") pod \"opendatahub-operator-controller-manager-77f6458f94-jpmrh\" (UID: \"0f1eb659-a77c-4638-8326-6be9dc29e08d\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6458f94-jpmrh" Nov 03 21:49:55.109267 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:55.109005 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0f1eb659-a77c-4638-8326-6be9dc29e08d-manifests\") pod \"opendatahub-operator-controller-manager-77f6458f94-jpmrh\" (UID: \"0f1eb659-a77c-4638-8326-6be9dc29e08d\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6458f94-jpmrh" Nov 03 21:49:55.109267 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:55.109033 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gltdm\" (UniqueName: \"kubernetes.io/projected/0f1eb659-a77c-4638-8326-6be9dc29e08d-kube-api-access-gltdm\") pod \"opendatahub-operator-controller-manager-77f6458f94-jpmrh\" (UID: \"0f1eb659-a77c-4638-8326-6be9dc29e08d\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6458f94-jpmrh" Nov 03 21:49:55.109267 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:55.109058 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0f1eb659-a77c-4638-8326-6be9dc29e08d-tmp\") pod \"opendatahub-operator-controller-manager-77f6458f94-jpmrh\" (UID: \"0f1eb659-a77c-4638-8326-6be9dc29e08d\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6458f94-jpmrh" Nov 03 21:49:55.109267 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:55.109083 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0f1eb659-a77c-4638-8326-6be9dc29e08d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-77f6458f94-jpmrh\" (UID: \"0f1eb659-a77c-4638-8326-6be9dc29e08d\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6458f94-jpmrh" Nov 03 21:49:55.109531 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:55.109492 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0f1eb659-a77c-4638-8326-6be9dc29e08d-manifests\") pod \"opendatahub-operator-controller-manager-77f6458f94-jpmrh\" (UID: \"0f1eb659-a77c-4638-8326-6be9dc29e08d\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6458f94-jpmrh" Nov 03 21:49:55.111532 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:55.111501 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0f1eb659-a77c-4638-8326-6be9dc29e08d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-77f6458f94-jpmrh\" (UID: \"0f1eb659-a77c-4638-8326-6be9dc29e08d\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6458f94-jpmrh" Nov 03 21:49:55.111532 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:55.111512 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0f1eb659-a77c-4638-8326-6be9dc29e08d-tmp\") pod \"opendatahub-operator-controller-manager-77f6458f94-jpmrh\" (UID: \"0f1eb659-a77c-4638-8326-6be9dc29e08d\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6458f94-jpmrh" Nov 03 21:49:55.111702 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:55.111669 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0f1eb659-a77c-4638-8326-6be9dc29e08d-webhook-cert\") pod \"opendatahub-operator-controller-manager-77f6458f94-jpmrh\" (UID: \"0f1eb659-a77c-4638-8326-6be9dc29e08d\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6458f94-jpmrh" Nov 03 21:49:55.111746 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:55.111705 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0f1eb659-a77c-4638-8326-6be9dc29e08d-metrics-cert\") pod \"opendatahub-operator-controller-manager-77f6458f94-jpmrh\" (UID: \"0f1eb659-a77c-4638-8326-6be9dc29e08d\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6458f94-jpmrh" Nov 03 21:49:55.121794 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:55.121766 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gltdm\" (UniqueName: \"kubernetes.io/projected/0f1eb659-a77c-4638-8326-6be9dc29e08d-kube-api-access-gltdm\") pod \"opendatahub-operator-controller-manager-77f6458f94-jpmrh\" (UID: \"0f1eb659-a77c-4638-8326-6be9dc29e08d\") " pod="opendatahub/opendatahub-operator-controller-manager-77f6458f94-jpmrh" Nov 03 21:49:55.168815 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:55.168777 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-77f6458f94-jpmrh" Nov 03 21:49:55.469697 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:55.469618 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-77f6458f94-jpmrh"] Nov 03 21:49:55.472038 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:49:55.471999 6524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f1eb659_a77c_4638_8326_6be9dc29e08d.slice/crio-ec3b53a023fab00b5774a00122a670b2418349a7139f6f5396edad243f0061bf WatchSource:0}: Error finding container ec3b53a023fab00b5774a00122a670b2418349a7139f6f5396edad243f0061bf: Status 404 returned error can't find the container with id ec3b53a023fab00b5774a00122a670b2418349a7139f6f5396edad243f0061bf Nov 03 21:49:55.652500 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:55.652467 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-77f6458f94-jpmrh" event={"ID":"0f1eb659-a77c-4638-8326-6be9dc29e08d","Type":"ContainerStarted","Data":"ec3b53a023fab00b5774a00122a670b2418349a7139f6f5396edad243f0061bf"} Nov 03 21:49:58.664312 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:58.664274 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-77f6458f94-jpmrh" event={"ID":"0f1eb659-a77c-4638-8326-6be9dc29e08d","Type":"ContainerStarted","Data":"c7532ea0833c67b37915c644417ae104a71fd74f177f6b6a558051a56279da2a"} Nov 03 21:49:59.669079 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:59.669041 6524 generic.go:356] "Generic (PLEG): container finished" podID="0f1eb659-a77c-4638-8326-6be9dc29e08d" containerID="c7532ea0833c67b37915c644417ae104a71fd74f177f6b6a558051a56279da2a" exitCode=0 Nov 03 21:49:59.669577 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:49:59.669100 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-77f6458f94-jpmrh" event={"ID":"0f1eb659-a77c-4638-8326-6be9dc29e08d","Type":"ContainerDied","Data":"c7532ea0833c67b37915c644417ae104a71fd74f177f6b6a558051a56279da2a"} Nov 03 21:49:59.706356 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:49:59.706320 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 21:50:00.674530 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:50:00.674492 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-77f6458f94-jpmrh" event={"ID":"0f1eb659-a77c-4638-8326-6be9dc29e08d","Type":"ContainerStarted","Data":"4afe210c4063ddcbc19107d5548cb466645d8445a33212f97d5e22816a7f8264"} Nov 03 21:50:00.675016 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:50:00.674607 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-77f6458f94-jpmrh" Nov 03 21:50:00.720039 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:50:00.719988 6524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-77f6458f94-jpmrh" podStartSLOduration=2.646961161 podStartE2EDuration="6.71997496s" totalImagesPullingTime="3.60951153s" totalInitContainerRuntime="463.502269ms" isStatefulPod=false podCreationTimestamp="2026-11-03 21:49:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:50:00.719514854 +0000 UTC m=+246.071841683" watchObservedRunningTime="2026-11-03 21:50:00.71997496 +0000 UTC m=+246.072301761" Nov 03 21:50:11.680840 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:50:11.680806 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-77f6458f94-jpmrh" Nov 03 21:50:27.851856 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:50:27.851823 6524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-module-controller-manager-5cfb77bf8c-hg589"] Nov 03 21:50:27.859775 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:50:27.859747 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-hg589" Nov 03 21:50:27.867755 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:50:27.867725 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-5cfb77bf8c-hg589"] Nov 03 21:50:27.875511 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:50:27.875488 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"odh-trusted-ca-bundle\"" Nov 03 21:50:27.879540 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:50:27.879515 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-module-controller-manager-dockercfg-2k7hr\"" Nov 03 21:50:27.962286 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:50:27.962244 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/38f1f654-c620-48a0-a6dc-748bbc212279-tmp\") pod \"kserve-module-controller-manager-5cfb77bf8c-hg589\" (UID: \"38f1f654-c620-48a0-a6dc-748bbc212279\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-hg589" Nov 03 21:50:27.962286 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:50:27.962289 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/38f1f654-c620-48a0-a6dc-748bbc212279-trusted-ca\") pod \"kserve-module-controller-manager-5cfb77bf8c-hg589\" (UID: \"38f1f654-c620-48a0-a6dc-748bbc212279\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-hg589" Nov 03 21:50:27.962499 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:50:27.962432 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5s9hn\" (UniqueName: \"kubernetes.io/projected/38f1f654-c620-48a0-a6dc-748bbc212279-kube-api-access-5s9hn\") pod \"kserve-module-controller-manager-5cfb77bf8c-hg589\" (UID: \"38f1f654-c620-48a0-a6dc-748bbc212279\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-hg589" Nov 03 21:50:27.962499 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:50:27.962472 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/38f1f654-c620-48a0-a6dc-748bbc212279-manifests\") pod \"kserve-module-controller-manager-5cfb77bf8c-hg589\" (UID: \"38f1f654-c620-48a0-a6dc-748bbc212279\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-hg589" Nov 03 21:50:28.063526 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:50:28.063484 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/38f1f654-c620-48a0-a6dc-748bbc212279-tmp\") pod \"kserve-module-controller-manager-5cfb77bf8c-hg589\" (UID: \"38f1f654-c620-48a0-a6dc-748bbc212279\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-hg589" Nov 03 21:50:28.063730 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:50:28.063530 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/38f1f654-c620-48a0-a6dc-748bbc212279-trusted-ca\") pod \"kserve-module-controller-manager-5cfb77bf8c-hg589\" (UID: \"38f1f654-c620-48a0-a6dc-748bbc212279\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-hg589" Nov 03 21:50:28.063730 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:50:28.063688 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5s9hn\" (UniqueName: \"kubernetes.io/projected/38f1f654-c620-48a0-a6dc-748bbc212279-kube-api-access-5s9hn\") pod \"kserve-module-controller-manager-5cfb77bf8c-hg589\" (UID: \"38f1f654-c620-48a0-a6dc-748bbc212279\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-hg589" Nov 03 21:50:28.063730 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:50:28.063721 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/38f1f654-c620-48a0-a6dc-748bbc212279-manifests\") pod \"kserve-module-controller-manager-5cfb77bf8c-hg589\" (UID: \"38f1f654-c620-48a0-a6dc-748bbc212279\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-hg589" Nov 03 21:50:28.063900 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:50:28.063863 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/38f1f654-c620-48a0-a6dc-748bbc212279-tmp\") pod \"kserve-module-controller-manager-5cfb77bf8c-hg589\" (UID: \"38f1f654-c620-48a0-a6dc-748bbc212279\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-hg589" Nov 03 21:50:28.064086 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:50:28.064065 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/38f1f654-c620-48a0-a6dc-748bbc212279-manifests\") pod \"kserve-module-controller-manager-5cfb77bf8c-hg589\" (UID: \"38f1f654-c620-48a0-a6dc-748bbc212279\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-hg589" Nov 03 21:50:28.064447 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:50:28.064428 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/38f1f654-c620-48a0-a6dc-748bbc212279-trusted-ca\") pod \"kserve-module-controller-manager-5cfb77bf8c-hg589\" (UID: \"38f1f654-c620-48a0-a6dc-748bbc212279\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-hg589" Nov 03 21:50:28.076586 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:50:28.076556 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5s9hn\" (UniqueName: \"kubernetes.io/projected/38f1f654-c620-48a0-a6dc-748bbc212279-kube-api-access-5s9hn\") pod \"kserve-module-controller-manager-5cfb77bf8c-hg589\" (UID: \"38f1f654-c620-48a0-a6dc-748bbc212279\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-hg589" Nov 03 21:50:28.172480 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:50:28.172385 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-hg589" Nov 03 21:50:28.444486 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:50:28.444407 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-5cfb77bf8c-hg589"] Nov 03 21:50:28.447456 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:50:28.447425 6524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38f1f654_c620_48a0_a6dc_748bbc212279.slice/crio-71e1acd8961185dd69db607443c6fe7c770f068841fe33bfcde9a740c5d953d6 WatchSource:0}: Error finding container 71e1acd8961185dd69db607443c6fe7c770f068841fe33bfcde9a740c5d953d6: Status 404 returned error can't find the container with id 71e1acd8961185dd69db607443c6fe7c770f068841fe33bfcde9a740c5d953d6 Nov 03 21:50:28.773465 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:50:28.773384 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-hg589" event={"ID":"38f1f654-c620-48a0-a6dc-748bbc212279","Type":"ContainerStarted","Data":"71e1acd8961185dd69db607443c6fe7c770f068841fe33bfcde9a740c5d953d6"} Nov 03 21:50:29.707296 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:50:29.707260 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 21:50:31.786420 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:50:31.786382 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-hg589" event={"ID":"38f1f654-c620-48a0-a6dc-748bbc212279","Type":"ContainerStarted","Data":"2bf1f9a8de838955e1bf3490429fbb3cd72c746bdd57afd0bf76c851ea1c19a0"} Nov 03 21:50:31.786912 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:50:31.786465 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-hg589" Nov 03 21:50:31.810047 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:50:31.809925 6524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-hg589" podStartSLOduration=2.298038703 podStartE2EDuration="4.809907796s" totalImagesPullingTime="2.511869093s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:50:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:50:31.809710215 +0000 UTC m=+277.162037047" watchObservedRunningTime="2026-11-03 21:50:31.809907796 +0000 UTC m=+277.162234600" Nov 03 21:50:52.792080 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:50:52.792041 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-hg589" Nov 03 21:50:54.786839 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:50:54.786810 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x26mj_76122905-15b4-4f4e-97b9-9f73a7675e36/ovn-acl-logging/0.log" Nov 03 21:50:54.788505 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:50:54.788480 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x26mj_76122905-15b4-4f4e-97b9-9f73a7675e36/ovn-acl-logging/0.log" Nov 03 21:50:59.707067 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:50:59.707024 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 21:51:13.549997 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:13.547842 6524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-ffrgp"] Nov 03 21:51:13.556641 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:13.556612 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ffrgp" Nov 03 21:51:13.560222 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:13.560192 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-ffrgp"] Nov 03 21:51:13.567660 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:13.567636 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-m5lxj\"" Nov 03 21:51:13.567781 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:13.567660 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Nov 03 21:51:13.638637 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:13.638602 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e4c25e3-c2ca-467c-8f78-a7af4841f67d-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-ffrgp\" (UID: \"1e4c25e3-c2ca-467c-8f78-a7af4841f67d\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ffrgp" Nov 03 21:51:13.638810 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:13.638650 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szcwf\" (UniqueName: \"kubernetes.io/projected/1e4c25e3-c2ca-467c-8f78-a7af4841f67d-kube-api-access-szcwf\") pod \"llmisvc-controller-manager-869d5dcc5d-ffrgp\" (UID: \"1e4c25e3-c2ca-467c-8f78-a7af4841f67d\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ffrgp" Nov 03 21:51:13.739792 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:13.739756 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e4c25e3-c2ca-467c-8f78-a7af4841f67d-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-ffrgp\" (UID: \"1e4c25e3-c2ca-467c-8f78-a7af4841f67d\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ffrgp" Nov 03 21:51:13.739980 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:13.739808 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szcwf\" (UniqueName: \"kubernetes.io/projected/1e4c25e3-c2ca-467c-8f78-a7af4841f67d-kube-api-access-szcwf\") pod \"llmisvc-controller-manager-869d5dcc5d-ffrgp\" (UID: \"1e4c25e3-c2ca-467c-8f78-a7af4841f67d\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ffrgp" Nov 03 21:51:13.742213 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:13.742182 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e4c25e3-c2ca-467c-8f78-a7af4841f67d-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-ffrgp\" (UID: \"1e4c25e3-c2ca-467c-8f78-a7af4841f67d\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ffrgp" Nov 03 21:51:13.755239 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:13.755206 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-szcwf\" (UniqueName: \"kubernetes.io/projected/1e4c25e3-c2ca-467c-8f78-a7af4841f67d-kube-api-access-szcwf\") pod \"llmisvc-controller-manager-869d5dcc5d-ffrgp\" (UID: \"1e4c25e3-c2ca-467c-8f78-a7af4841f67d\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ffrgp" Nov 03 21:51:13.783748 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:13.783711 6524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7vrmd"] Nov 03 21:51:13.787118 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:13.787095 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7vrmd" Nov 03 21:51:13.794767 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:13.794741 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Nov 03 21:51:13.794889 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:13.794832 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Nov 03 21:51:13.794889 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:13.794836 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Nov 03 21:51:13.795750 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:13.795727 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Nov 03 21:51:13.795863 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:13.795812 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Nov 03 21:51:13.795863 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:13.795822 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Nov 03 21:51:13.795973 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:13.795904 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-7h8b9\"" Nov 03 21:51:13.800402 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:13.800360 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7vrmd"] Nov 03 21:51:13.840550 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:13.840516 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gffpb\" (UniqueName: \"kubernetes.io/projected/497ce0ef-46c2-4727-8424-e676f303bde5-kube-api-access-gffpb\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7vrmd\" (UID: \"497ce0ef-46c2-4727-8424-e676f303bde5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7vrmd" Nov 03 21:51:13.840703 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:13.840555 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/497ce0ef-46c2-4727-8424-e676f303bde5-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7vrmd\" (UID: \"497ce0ef-46c2-4727-8424-e676f303bde5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7vrmd" Nov 03 21:51:13.840703 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:13.840673 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/497ce0ef-46c2-4727-8424-e676f303bde5-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7vrmd\" (UID: \"497ce0ef-46c2-4727-8424-e676f303bde5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7vrmd" Nov 03 21:51:13.840811 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:13.840708 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/497ce0ef-46c2-4727-8424-e676f303bde5-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7vrmd\" (UID: \"497ce0ef-46c2-4727-8424-e676f303bde5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7vrmd" Nov 03 21:51:13.840811 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:13.840728 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/497ce0ef-46c2-4727-8424-e676f303bde5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7vrmd\" (UID: \"497ce0ef-46c2-4727-8424-e676f303bde5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7vrmd" Nov 03 21:51:13.840886 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:13.840852 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/497ce0ef-46c2-4727-8424-e676f303bde5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7vrmd\" (UID: \"497ce0ef-46c2-4727-8424-e676f303bde5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7vrmd" Nov 03 21:51:13.840975 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:13.840959 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/497ce0ef-46c2-4727-8424-e676f303bde5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7vrmd\" (UID: \"497ce0ef-46c2-4727-8424-e676f303bde5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7vrmd" Nov 03 21:51:13.871610 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:13.871576 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ffrgp" Nov 03 21:51:13.941504 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:13.941476 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gffpb\" (UniqueName: \"kubernetes.io/projected/497ce0ef-46c2-4727-8424-e676f303bde5-kube-api-access-gffpb\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7vrmd\" (UID: \"497ce0ef-46c2-4727-8424-e676f303bde5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7vrmd" Nov 03 21:51:13.941672 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:13.941515 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/497ce0ef-46c2-4727-8424-e676f303bde5-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7vrmd\" (UID: \"497ce0ef-46c2-4727-8424-e676f303bde5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7vrmd" Nov 03 21:51:13.941672 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:13.941549 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/497ce0ef-46c2-4727-8424-e676f303bde5-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7vrmd\" (UID: \"497ce0ef-46c2-4727-8424-e676f303bde5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7vrmd" Nov 03 21:51:13.941672 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:13.941574 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/497ce0ef-46c2-4727-8424-e676f303bde5-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7vrmd\" (UID: \"497ce0ef-46c2-4727-8424-e676f303bde5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7vrmd" Nov 03 21:51:13.941672 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:13.941602 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/497ce0ef-46c2-4727-8424-e676f303bde5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7vrmd\" (UID: \"497ce0ef-46c2-4727-8424-e676f303bde5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7vrmd" Nov 03 21:51:13.941672 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:13.941657 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/497ce0ef-46c2-4727-8424-e676f303bde5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7vrmd\" (UID: \"497ce0ef-46c2-4727-8424-e676f303bde5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7vrmd" Nov 03 21:51:13.941923 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:13.941685 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/497ce0ef-46c2-4727-8424-e676f303bde5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7vrmd\" (UID: \"497ce0ef-46c2-4727-8424-e676f303bde5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7vrmd" Nov 03 21:51:13.942830 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:13.942798 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/497ce0ef-46c2-4727-8424-e676f303bde5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7vrmd\" (UID: \"497ce0ef-46c2-4727-8424-e676f303bde5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7vrmd" Nov 03 21:51:13.944536 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:13.944516 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/497ce0ef-46c2-4727-8424-e676f303bde5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7vrmd\" (UID: \"497ce0ef-46c2-4727-8424-e676f303bde5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7vrmd" Nov 03 21:51:13.944807 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:13.944770 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/497ce0ef-46c2-4727-8424-e676f303bde5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7vrmd\" (UID: \"497ce0ef-46c2-4727-8424-e676f303bde5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7vrmd" Nov 03 21:51:13.944807 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:13.944789 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/497ce0ef-46c2-4727-8424-e676f303bde5-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7vrmd\" (UID: \"497ce0ef-46c2-4727-8424-e676f303bde5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7vrmd" Nov 03 21:51:13.949917 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:13.949895 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/497ce0ef-46c2-4727-8424-e676f303bde5-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7vrmd\" (UID: \"497ce0ef-46c2-4727-8424-e676f303bde5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7vrmd" Nov 03 21:51:13.956970 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:13.956911 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/497ce0ef-46c2-4727-8424-e676f303bde5-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7vrmd\" (UID: \"497ce0ef-46c2-4727-8424-e676f303bde5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7vrmd" Nov 03 21:51:13.957063 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:13.957042 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gffpb\" (UniqueName: \"kubernetes.io/projected/497ce0ef-46c2-4727-8424-e676f303bde5-kube-api-access-gffpb\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7vrmd\" (UID: \"497ce0ef-46c2-4727-8424-e676f303bde5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7vrmd" Nov 03 21:51:14.098079 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:14.098038 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7vrmd" Nov 03 21:51:14.142873 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:14.142837 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-ffrgp"] Nov 03 21:51:14.147035 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:51:14.147007 6524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1e4c25e3_c2ca_467c_8f78_a7af4841f67d.slice/crio-93c77beea52b6e8618d82ff0d9c79421434fcbe230d71918298f0a311e6de445 WatchSource:0}: Error finding container 93c77beea52b6e8618d82ff0d9c79421434fcbe230d71918298f0a311e6de445: Status 404 returned error can't find the container with id 93c77beea52b6e8618d82ff0d9c79421434fcbe230d71918298f0a311e6de445 Nov 03 21:51:14.376722 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:14.376630 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7vrmd"] Nov 03 21:51:14.379602 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:51:14.379572 6524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod497ce0ef_46c2_4727_8424_e676f303bde5.slice/crio-a17cac568cd7bdc0d0f17c7d90c1392e2f2dca23cbc34aebc71505882c139dcc WatchSource:0}: Error finding container a17cac568cd7bdc0d0f17c7d90c1392e2f2dca23cbc34aebc71505882c139dcc: Status 404 returned error can't find the container with id a17cac568cd7bdc0d0f17c7d90c1392e2f2dca23cbc34aebc71505882c139dcc Nov 03 21:51:14.944872 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:14.944832 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ffrgp" event={"ID":"1e4c25e3-c2ca-467c-8f78-a7af4841f67d","Type":"ContainerStarted","Data":"93c77beea52b6e8618d82ff0d9c79421434fcbe230d71918298f0a311e6de445"} Nov 03 21:51:14.945997 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:14.945971 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7vrmd" event={"ID":"497ce0ef-46c2-4727-8424-e676f303bde5","Type":"ContainerStarted","Data":"a17cac568cd7bdc0d0f17c7d90c1392e2f2dca23cbc34aebc71505882c139dcc"} Nov 03 21:51:17.488809 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:17.488400 6524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"} Nov 03 21:51:17.488809 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:17.488482 6524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"} Nov 03 21:51:17.960801 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:17.960759 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ffrgp" event={"ID":"1e4c25e3-c2ca-467c-8f78-a7af4841f67d","Type":"ContainerStarted","Data":"ba61395e2428b3792659f725a332c323d8602317d9a5cf336d2ff98d9bd71584"} Nov 03 21:51:17.961004 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:17.960893 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ffrgp" Nov 03 21:51:17.962396 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:17.962370 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7vrmd" event={"ID":"497ce0ef-46c2-4727-8424-e676f303bde5","Type":"ContainerStarted","Data":"095cf3ee7b821e7839781fad9ad2a29294aff18877876a6a7d3af89f871ed5d5"} Nov 03 21:51:17.962522 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:17.962508 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7vrmd" Nov 03 21:51:17.995505 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:17.995450 6524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ffrgp" podStartSLOduration=1.504966284 podStartE2EDuration="4.995436898s" totalImagesPullingTime="3.490470614s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:51:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:51:17.994335302 +0000 UTC m=+323.346662106" watchObservedRunningTime="2026-11-03 21:51:17.995436898 +0000 UTC m=+323.347763693" Nov 03 21:51:18.018650 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:18.018597 6524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7vrmd" podStartSLOduration=1.91275174 podStartE2EDuration="5.018574899s" totalImagesPullingTime="3.105823159s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:51:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:51:18.018173215 +0000 UTC m=+323.370500018" watchObservedRunningTime="2026-11-03 21:51:18.018574899 +0000 UTC m=+323.370901703" Nov 03 21:51:18.967341 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:18.967311 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7vrmd" Nov 03 21:51:23.550106 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:23.550068 6524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6c6f488f88-7rl88"] Nov 03 21:51:23.553497 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:23.553470 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6c6f488f88-7rl88" Nov 03 21:51:23.566665 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:23.566634 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6c6f488f88-7rl88"] Nov 03 21:51:23.604813 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:23.604779 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Nov 03 21:51:23.631197 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:23.631151 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f614ff28-8d30-47e9-a78f-cd2078f2d35c-metrics-cert\") pod \"lws-controller-manager-6c6f488f88-7rl88\" (UID: \"f614ff28-8d30-47e9-a78f-cd2078f2d35c\") " pod="openshift-lws-operator/lws-controller-manager-6c6f488f88-7rl88" Nov 03 21:51:23.631370 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:23.631290 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f614ff28-8d30-47e9-a78f-cd2078f2d35c-cert\") pod \"lws-controller-manager-6c6f488f88-7rl88\" (UID: \"f614ff28-8d30-47e9-a78f-cd2078f2d35c\") " pod="openshift-lws-operator/lws-controller-manager-6c6f488f88-7rl88" Nov 03 21:51:23.631370 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:23.631333 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f614ff28-8d30-47e9-a78f-cd2078f2d35c-manager-config\") pod \"lws-controller-manager-6c6f488f88-7rl88\" (UID: \"f614ff28-8d30-47e9-a78f-cd2078f2d35c\") " pod="openshift-lws-operator/lws-controller-manager-6c6f488f88-7rl88" Nov 03 21:51:23.631370 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:23.631350 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqmxq\" (UniqueName: \"kubernetes.io/projected/f614ff28-8d30-47e9-a78f-cd2078f2d35c-kube-api-access-rqmxq\") pod \"lws-controller-manager-6c6f488f88-7rl88\" (UID: \"f614ff28-8d30-47e9-a78f-cd2078f2d35c\") " pod="openshift-lws-operator/lws-controller-manager-6c6f488f88-7rl88" Nov 03 21:51:23.652752 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:23.652722 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Nov 03 21:51:23.652752 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:23.652723 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Nov 03 21:51:23.652900 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:23.652803 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-srjhq\"" Nov 03 21:51:23.732713 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:23.732669 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f614ff28-8d30-47e9-a78f-cd2078f2d35c-manager-config\") pod \"lws-controller-manager-6c6f488f88-7rl88\" (UID: \"f614ff28-8d30-47e9-a78f-cd2078f2d35c\") " pod="openshift-lws-operator/lws-controller-manager-6c6f488f88-7rl88" Nov 03 21:51:23.732713 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:23.732712 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqmxq\" (UniqueName: \"kubernetes.io/projected/f614ff28-8d30-47e9-a78f-cd2078f2d35c-kube-api-access-rqmxq\") pod \"lws-controller-manager-6c6f488f88-7rl88\" (UID: \"f614ff28-8d30-47e9-a78f-cd2078f2d35c\") " pod="openshift-lws-operator/lws-controller-manager-6c6f488f88-7rl88" Nov 03 21:51:23.732967 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:23.732743 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f614ff28-8d30-47e9-a78f-cd2078f2d35c-metrics-cert\") pod \"lws-controller-manager-6c6f488f88-7rl88\" (UID: \"f614ff28-8d30-47e9-a78f-cd2078f2d35c\") " pod="openshift-lws-operator/lws-controller-manager-6c6f488f88-7rl88" Nov 03 21:51:23.732967 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:23.732909 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f614ff28-8d30-47e9-a78f-cd2078f2d35c-cert\") pod \"lws-controller-manager-6c6f488f88-7rl88\" (UID: \"f614ff28-8d30-47e9-a78f-cd2078f2d35c\") " pod="openshift-lws-operator/lws-controller-manager-6c6f488f88-7rl88" Nov 03 21:51:23.733514 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:23.733486 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f614ff28-8d30-47e9-a78f-cd2078f2d35c-manager-config\") pod \"lws-controller-manager-6c6f488f88-7rl88\" (UID: \"f614ff28-8d30-47e9-a78f-cd2078f2d35c\") " pod="openshift-lws-operator/lws-controller-manager-6c6f488f88-7rl88" Nov 03 21:51:23.735988 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:23.735963 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f614ff28-8d30-47e9-a78f-cd2078f2d35c-metrics-cert\") pod \"lws-controller-manager-6c6f488f88-7rl88\" (UID: \"f614ff28-8d30-47e9-a78f-cd2078f2d35c\") " pod="openshift-lws-operator/lws-controller-manager-6c6f488f88-7rl88" Nov 03 21:51:23.736494 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:23.736472 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f614ff28-8d30-47e9-a78f-cd2078f2d35c-cert\") pod \"lws-controller-manager-6c6f488f88-7rl88\" (UID: \"f614ff28-8d30-47e9-a78f-cd2078f2d35c\") " pod="openshift-lws-operator/lws-controller-manager-6c6f488f88-7rl88" Nov 03 21:51:23.745183 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:23.745156 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqmxq\" (UniqueName: \"kubernetes.io/projected/f614ff28-8d30-47e9-a78f-cd2078f2d35c-kube-api-access-rqmxq\") pod \"lws-controller-manager-6c6f488f88-7rl88\" (UID: \"f614ff28-8d30-47e9-a78f-cd2078f2d35c\") " pod="openshift-lws-operator/lws-controller-manager-6c6f488f88-7rl88" Nov 03 21:51:23.865342 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:23.865276 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6c6f488f88-7rl88" Nov 03 21:51:24.136822 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:24.136746 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6c6f488f88-7rl88"] Nov 03 21:51:24.140497 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:51:24.140463 6524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf614ff28_8d30_47e9_a78f_cd2078f2d35c.slice/crio-edef78d0038415f94e5462f477d9986f7ad973414573de0f84a75eea299dd57a WatchSource:0}: Error finding container edef78d0038415f94e5462f477d9986f7ad973414573de0f84a75eea299dd57a: Status 404 returned error can't find the container with id edef78d0038415f94e5462f477d9986f7ad973414573de0f84a75eea299dd57a Nov 03 21:51:24.988239 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:24.988204 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6c6f488f88-7rl88" event={"ID":"f614ff28-8d30-47e9-a78f-cd2078f2d35c","Type":"ContainerStarted","Data":"edef78d0038415f94e5462f477d9986f7ad973414573de0f84a75eea299dd57a"} Nov 03 21:51:26.998289 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:26.998249 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6c6f488f88-7rl88" event={"ID":"f614ff28-8d30-47e9-a78f-cd2078f2d35c","Type":"ContainerStarted","Data":"7a15c3d92632a74c812c578bf65ee5c59181b384fe3966ccb55764f29f75a182"} Nov 03 21:51:26.998759 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:26.998366 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6c6f488f88-7rl88" Nov 03 21:51:27.018947 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:27.018895 6524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6c6f488f88-7rl88" podStartSLOduration=2.032554532 podStartE2EDuration="4.018879864s" totalImagesPullingTime="1.986325332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:51:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:51:27.018162164 +0000 UTC m=+332.370488969" watchObservedRunningTime="2026-11-03 21:51:27.018879864 +0000 UTC m=+332.371206667" Nov 03 21:51:29.706944 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:51:29.706903 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 21:51:38.005675 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:38.005636 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6c6f488f88-7rl88" Nov 03 21:51:48.967608 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:51:48.967577 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ffrgp" Nov 03 21:51:59.706744 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:51:59.706704 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 21:52:29.707796 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:52:29.707752 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 21:52:59.707785 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:52:59.707748 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 21:53:29.707626 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:53:29.707577 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 21:53:53.462160 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:53:53.462111 6524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-vkpbm"] Nov 03 21:53:53.465458 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:53:53.465436 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-vkpbm" Nov 03 21:53:53.469379 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:53:53.469356 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Nov 03 21:53:53.469493 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:53:53.469397 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Nov 03 21:53:53.477157 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:53:53.474948 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-h56fq\"" Nov 03 21:53:53.477157 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:53:53.476273 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-vkpbm"] Nov 03 21:53:53.569611 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:53:53.569583 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Nov 03 21:53:53.578012 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:53:53.577980 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4844f9ef-f2e3-4483-ae7a-fcb57a4ded1e-data\") pod \"postgres-7cdbf6dc44-vkpbm\" (UID: \"4844f9ef-f2e3-4483-ae7a-fcb57a4ded1e\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-vkpbm" Nov 03 21:53:53.578184 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:53:53.578025 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dd86m\" (UniqueName: \"kubernetes.io/projected/4844f9ef-f2e3-4483-ae7a-fcb57a4ded1e-kube-api-access-dd86m\") pod \"postgres-7cdbf6dc44-vkpbm\" (UID: \"4844f9ef-f2e3-4483-ae7a-fcb57a4ded1e\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-vkpbm" Nov 03 21:53:53.679426 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:53:53.679379 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4844f9ef-f2e3-4483-ae7a-fcb57a4ded1e-data\") pod \"postgres-7cdbf6dc44-vkpbm\" (UID: \"4844f9ef-f2e3-4483-ae7a-fcb57a4ded1e\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-vkpbm" Nov 03 21:53:53.679608 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:53:53.679436 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd86m\" (UniqueName: \"kubernetes.io/projected/4844f9ef-f2e3-4483-ae7a-fcb57a4ded1e-kube-api-access-dd86m\") pod \"postgres-7cdbf6dc44-vkpbm\" (UID: \"4844f9ef-f2e3-4483-ae7a-fcb57a4ded1e\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-vkpbm" Nov 03 21:53:53.679773 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:53:53.679752 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4844f9ef-f2e3-4483-ae7a-fcb57a4ded1e-data\") pod \"postgres-7cdbf6dc44-vkpbm\" (UID: \"4844f9ef-f2e3-4483-ae7a-fcb57a4ded1e\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-vkpbm" Nov 03 21:53:53.693011 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:53:53.692975 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd86m\" (UniqueName: \"kubernetes.io/projected/4844f9ef-f2e3-4483-ae7a-fcb57a4ded1e-kube-api-access-dd86m\") pod \"postgres-7cdbf6dc44-vkpbm\" (UID: \"4844f9ef-f2e3-4483-ae7a-fcb57a4ded1e\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-vkpbm" Nov 03 21:53:53.780442 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:53:53.780357 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-vkpbm" Nov 03 21:53:54.051380 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:53:54.051298 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-vkpbm"] Nov 03 21:53:54.054689 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:53:54.054654 6524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4844f9ef_f2e3_4483_ae7a_fcb57a4ded1e.slice/crio-a0db501f9df372993e39f01a19c8fc3c64165552254b7068f84967778e64af47 WatchSource:0}: Error finding container a0db501f9df372993e39f01a19c8fc3c64165552254b7068f84967778e64af47: Status 404 returned error can't find the container with id a0db501f9df372993e39f01a19c8fc3c64165552254b7068f84967778e64af47 Nov 03 21:53:54.564588 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:53:54.564553 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-vkpbm" event={"ID":"4844f9ef-f2e3-4483-ae7a-fcb57a4ded1e","Type":"ContainerStarted","Data":"a0db501f9df372993e39f01a19c8fc3c64165552254b7068f84967778e64af47"} Nov 03 21:53:59.473539 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:53:59.473510 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Nov 03 21:53:59.588448 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:53:59.588401 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-vkpbm" event={"ID":"4844f9ef-f2e3-4483-ae7a-fcb57a4ded1e","Type":"ContainerStarted","Data":"4cd3c87d6de19325350c8b4908ba1f8cedc0f467482bf4fa5ae5089c4ae714db"} Nov 03 21:53:59.588653 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:53:59.588632 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-vkpbm" Nov 03 21:53:59.607420 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:53:59.607364 6524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-vkpbm" podStartSLOduration=1.193287753 podStartE2EDuration="6.607348732s" totalImagesPullingTime="5.414060979s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:53:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:53:59.606903232 +0000 UTC m=+484.959230037" watchObservedRunningTime="2026-11-03 21:53:59.607348732 +0000 UTC m=+484.959675535" Nov 03 21:53:59.707610 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:53:59.707564 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 21:54:05.621850 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:54:05.621819 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-vkpbm" Nov 03 21:54:29.707800 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:54:29.707754 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 21:54:59.706731 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:54:59.706687 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 21:55:05.793069 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:55:05.793030 6524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-k99zx"] Nov 03 21:55:05.796038 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:55:05.796019 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-k99zx" Nov 03 21:55:05.806562 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:55:05.806533 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-k99zx"] Nov 03 21:55:05.863040 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:55:05.863002 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d829g\" (UniqueName: \"kubernetes.io/projected/8559b8d0-420f-4b4c-a430-bb54f19c0fa6-kube-api-access-d829g\") pod \"llmisvc-controller-manager-6466c554d8-k99zx\" (UID: \"8559b8d0-420f-4b4c-a430-bb54f19c0fa6\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-k99zx" Nov 03 21:55:05.863040 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:55:05.863055 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8559b8d0-420f-4b4c-a430-bb54f19c0fa6-cert\") pod \"llmisvc-controller-manager-6466c554d8-k99zx\" (UID: \"8559b8d0-420f-4b4c-a430-bb54f19c0fa6\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-k99zx" Nov 03 21:55:05.964088 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:55:05.964052 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d829g\" (UniqueName: \"kubernetes.io/projected/8559b8d0-420f-4b4c-a430-bb54f19c0fa6-kube-api-access-d829g\") pod \"llmisvc-controller-manager-6466c554d8-k99zx\" (UID: \"8559b8d0-420f-4b4c-a430-bb54f19c0fa6\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-k99zx" Nov 03 21:55:05.964226 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:55:05.964106 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8559b8d0-420f-4b4c-a430-bb54f19c0fa6-cert\") pod \"llmisvc-controller-manager-6466c554d8-k99zx\" (UID: \"8559b8d0-420f-4b4c-a430-bb54f19c0fa6\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-k99zx" Nov 03 21:55:05.966541 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:55:05.966513 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8559b8d0-420f-4b4c-a430-bb54f19c0fa6-cert\") pod \"llmisvc-controller-manager-6466c554d8-k99zx\" (UID: \"8559b8d0-420f-4b4c-a430-bb54f19c0fa6\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-k99zx" Nov 03 21:55:05.977509 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:55:05.977482 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d829g\" (UniqueName: \"kubernetes.io/projected/8559b8d0-420f-4b4c-a430-bb54f19c0fa6-kube-api-access-d829g\") pod \"llmisvc-controller-manager-6466c554d8-k99zx\" (UID: \"8559b8d0-420f-4b4c-a430-bb54f19c0fa6\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-k99zx" Nov 03 21:55:06.108878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:55:06.108842 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-k99zx" Nov 03 21:55:06.381072 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:55:06.380995 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-k99zx"] Nov 03 21:55:06.384321 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:55:06.384291 6524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8559b8d0_420f_4b4c_a430_bb54f19c0fa6.slice/crio-bbc780833f2d8181f5211164f810eec80a1ae92b79255a42d25a7eae43e95976 WatchSource:0}: Error finding container bbc780833f2d8181f5211164f810eec80a1ae92b79255a42d25a7eae43e95976: Status 404 returned error can't find the container with id bbc780833f2d8181f5211164f810eec80a1ae92b79255a42d25a7eae43e95976 Nov 03 21:55:06.385669 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:55:06.385644 6524 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 03 21:55:06.852918 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:55:06.852881 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-k99zx" event={"ID":"8559b8d0-420f-4b4c-a430-bb54f19c0fa6","Type":"ContainerStarted","Data":"3f2f60fb5a88d63690e6621b22a94b7e1c037613d89442b278712dd0927b86d2"} Nov 03 21:55:06.853411 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:55:06.852937 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-k99zx" event={"ID":"8559b8d0-420f-4b4c-a430-bb54f19c0fa6","Type":"ContainerStarted","Data":"bbc780833f2d8181f5211164f810eec80a1ae92b79255a42d25a7eae43e95976"} Nov 03 21:55:06.853411 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:55:06.852965 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-k99zx" Nov 03 21:55:06.872235 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:55:06.872183 6524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-6466c554d8-k99zx" podStartSLOduration=1.476569914 podStartE2EDuration="1.872167939s" totalImagesPullingTime="395.598025ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:55:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:55:06.871910442 +0000 UTC m=+552.224237258" watchObservedRunningTime="2026-11-03 21:55:06.872167939 +0000 UTC m=+552.224494741" Nov 03 21:55:29.707535 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:55:29.707503 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 21:55:37.858527 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:55:37.858497 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-k99zx" Nov 03 21:55:37.909622 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:55:37.909586 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-ffrgp"] Nov 03 21:55:37.909824 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:55:37.909800 6524 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ffrgp" podUID="1e4c25e3-c2ca-467c-8f78-a7af4841f67d" containerName="manager" containerID="cri-o://ba61395e2428b3792659f725a332c323d8602317d9a5cf336d2ff98d9bd71584" gracePeriod=30 Nov 03 21:55:38.155936 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:55:38.155910 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ffrgp" Nov 03 21:55:38.349927 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:55:38.349891 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1e4c25e3-c2ca-467c-8f78-a7af4841f67d-cert\" (UniqueName: \"kubernetes.io/secret/1e4c25e3-c2ca-467c-8f78-a7af4841f67d-cert\") pod \"1e4c25e3-c2ca-467c-8f78-a7af4841f67d\" (UID: \"1e4c25e3-c2ca-467c-8f78-a7af4841f67d\") " Nov 03 21:55:38.350076 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:55:38.349960 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1e4c25e3-c2ca-467c-8f78-a7af4841f67d-kube-api-access-szcwf\" (UniqueName: \"kubernetes.io/projected/1e4c25e3-c2ca-467c-8f78-a7af4841f67d-kube-api-access-szcwf\") pod \"1e4c25e3-c2ca-467c-8f78-a7af4841f67d\" (UID: \"1e4c25e3-c2ca-467c-8f78-a7af4841f67d\") " Nov 03 21:55:38.352041 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:55:38.352009 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e4c25e3-c2ca-467c-8f78-a7af4841f67d-cert" pod "1e4c25e3-c2ca-467c-8f78-a7af4841f67d" (UID: "1e4c25e3-c2ca-467c-8f78-a7af4841f67d"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 03 21:55:38.352195 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:55:38.352064 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e4c25e3-c2ca-467c-8f78-a7af4841f67d-kube-api-access-szcwf" pod "1e4c25e3-c2ca-467c-8f78-a7af4841f67d" (UID: "1e4c25e3-c2ca-467c-8f78-a7af4841f67d"). InnerVolumeSpecName "kube-api-access-szcwf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 03 21:55:38.450919 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:55:38.450837 6524 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e4c25e3-c2ca-467c-8f78-a7af4841f67d-cert\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:55:38.450919 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:55:38.450867 6524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szcwf\" (UniqueName: \"kubernetes.io/projected/1e4c25e3-c2ca-467c-8f78-a7af4841f67d-kube-api-access-szcwf\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:55:38.980597 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:55:38.980557 6524 generic.go:356] "Generic (PLEG): container finished" podID="1e4c25e3-c2ca-467c-8f78-a7af4841f67d" containerID="ba61395e2428b3792659f725a332c323d8602317d9a5cf336d2ff98d9bd71584" exitCode=0 Nov 03 21:55:38.980597 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:55:38.980597 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ffrgp" event={"ID":"1e4c25e3-c2ca-467c-8f78-a7af4841f67d","Type":"ContainerDied","Data":"ba61395e2428b3792659f725a332c323d8602317d9a5cf336d2ff98d9bd71584"} Nov 03 21:55:38.981161 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:55:38.980625 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ffrgp" Nov 03 21:55:38.981161 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:55:38.980660 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ffrgp" event={"ID":"1e4c25e3-c2ca-467c-8f78-a7af4841f67d","Type":"ContainerDied","Data":"93c77beea52b6e8618d82ff0d9c79421434fcbe230d71918298f0a311e6de445"} Nov 03 21:55:38.981161 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:55:38.980682 6524 scope.go:122] "RemoveContainer" containerID="ba61395e2428b3792659f725a332c323d8602317d9a5cf336d2ff98d9bd71584" Nov 03 21:55:38.988753 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:55:38.988733 6524 scope.go:122] "RemoveContainer" containerID="ba61395e2428b3792659f725a332c323d8602317d9a5cf336d2ff98d9bd71584" Nov 03 21:55:38.989026 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:55:38.989006 6524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba61395e2428b3792659f725a332c323d8602317d9a5cf336d2ff98d9bd71584\": container with ID starting with ba61395e2428b3792659f725a332c323d8602317d9a5cf336d2ff98d9bd71584 not found: ID does not exist" containerID="ba61395e2428b3792659f725a332c323d8602317d9a5cf336d2ff98d9bd71584" Nov 03 21:55:38.989092 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:55:38.989041 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba61395e2428b3792659f725a332c323d8602317d9a5cf336d2ff98d9bd71584"} err="failed to get container status \"ba61395e2428b3792659f725a332c323d8602317d9a5cf336d2ff98d9bd71584\": rpc error: code = NotFound desc = could not find container \"ba61395e2428b3792659f725a332c323d8602317d9a5cf336d2ff98d9bd71584\": container with ID starting with ba61395e2428b3792659f725a332c323d8602317d9a5cf336d2ff98d9bd71584 not found: ID does not exist" Nov 03 21:55:39.011585 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:55:39.011550 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-ffrgp"] Nov 03 21:55:39.017850 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:55:39.017825 6524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-ffrgp"] Nov 03 21:55:40.766390 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:55:40.766347 6524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1e4c25e3-c2ca-467c-8f78-a7af4841f67d" path="/var/lib/kubelet/pods/1e4c25e3-c2ca-467c-8f78-a7af4841f67d/volumes" Nov 03 21:55:54.811109 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:55:54.811081 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x26mj_76122905-15b4-4f4e-97b9-9f73a7675e36/ovn-acl-logging/0.log" Nov 03 21:55:54.814729 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:55:54.814702 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x26mj_76122905-15b4-4f4e-97b9-9f73a7675e36/ovn-acl-logging/0.log" Nov 03 21:55:59.707286 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:55:59.707245 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 21:56:29.707834 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:56:29.707789 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 21:56:33.896582 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:33.896519 6524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp"] Nov 03 21:56:33.897171 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:33.896987 6524 cpu_manager.go:414] "containerMap: removing container" podUID="1e4c25e3-c2ca-467c-8f78-a7af4841f67d" containerName="manager" Nov 03 21:56:33.897171 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:33.897006 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1e4c25e3-c2ca-467c-8f78-a7af4841f67d" containerName="manager" Nov 03 21:56:33.897171 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:33.897107 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="1e4c25e3-c2ca-467c-8f78-a7af4841f67d" containerName="manager" Nov 03 21:56:33.900463 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:33.900438 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp" Nov 03 21:56:33.912280 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:33.912247 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp"] Nov 03 21:56:33.996147 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:33.996099 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-tn75j\"" Nov 03 21:56:34.013480 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:34.013448 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a85e1f40-b852-4024-a1eb-0c5798b081e7-istio-envoy\") pod \"maas-default-gateway-openshift-default-66db998887-2jqqp\" (UID: \"a85e1f40-b852-4024-a1eb-0c5798b081e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp" Nov 03 21:56:34.013624 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:34.013489 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a85e1f40-b852-4024-a1eb-0c5798b081e7-credential-socket\") pod \"maas-default-gateway-openshift-default-66db998887-2jqqp\" (UID: \"a85e1f40-b852-4024-a1eb-0c5798b081e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp" Nov 03 21:56:34.013624 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:34.013516 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vfxk\" (UniqueName: \"kubernetes.io/projected/a85e1f40-b852-4024-a1eb-0c5798b081e7-kube-api-access-6vfxk\") pod \"maas-default-gateway-openshift-default-66db998887-2jqqp\" (UID: \"a85e1f40-b852-4024-a1eb-0c5798b081e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp" Nov 03 21:56:34.013703 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:34.013654 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a85e1f40-b852-4024-a1eb-0c5798b081e7-istio-data\") pod \"maas-default-gateway-openshift-default-66db998887-2jqqp\" (UID: \"a85e1f40-b852-4024-a1eb-0c5798b081e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp" Nov 03 21:56:34.013703 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:34.013696 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a85e1f40-b852-4024-a1eb-0c5798b081e7-workload-socket\") pod \"maas-default-gateway-openshift-default-66db998887-2jqqp\" (UID: \"a85e1f40-b852-4024-a1eb-0c5798b081e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp" Nov 03 21:56:34.013773 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:34.013718 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a85e1f40-b852-4024-a1eb-0c5798b081e7-istio-token\") pod \"maas-default-gateway-openshift-default-66db998887-2jqqp\" (UID: \"a85e1f40-b852-4024-a1eb-0c5798b081e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp" Nov 03 21:56:34.013877 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:34.013848 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a85e1f40-b852-4024-a1eb-0c5798b081e7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-66db998887-2jqqp\" (UID: \"a85e1f40-b852-4024-a1eb-0c5798b081e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp" Nov 03 21:56:34.013928 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:34.013904 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a85e1f40-b852-4024-a1eb-0c5798b081e7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-66db998887-2jqqp\" (UID: \"a85e1f40-b852-4024-a1eb-0c5798b081e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp" Nov 03 21:56:34.014031 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:34.014009 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a85e1f40-b852-4024-a1eb-0c5798b081e7-workload-certs\") pod \"maas-default-gateway-openshift-default-66db998887-2jqqp\" (UID: \"a85e1f40-b852-4024-a1eb-0c5798b081e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp" Nov 03 21:56:34.115266 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:34.115217 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a85e1f40-b852-4024-a1eb-0c5798b081e7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-66db998887-2jqqp\" (UID: \"a85e1f40-b852-4024-a1eb-0c5798b081e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp" Nov 03 21:56:34.115477 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:34.115277 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a85e1f40-b852-4024-a1eb-0c5798b081e7-workload-certs\") pod \"maas-default-gateway-openshift-default-66db998887-2jqqp\" (UID: \"a85e1f40-b852-4024-a1eb-0c5798b081e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp" Nov 03 21:56:34.115477 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:34.115335 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a85e1f40-b852-4024-a1eb-0c5798b081e7-istio-envoy\") pod \"maas-default-gateway-openshift-default-66db998887-2jqqp\" (UID: \"a85e1f40-b852-4024-a1eb-0c5798b081e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp" Nov 03 21:56:34.115477 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:34.115372 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a85e1f40-b852-4024-a1eb-0c5798b081e7-credential-socket\") pod \"maas-default-gateway-openshift-default-66db998887-2jqqp\" (UID: \"a85e1f40-b852-4024-a1eb-0c5798b081e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp" Nov 03 21:56:34.115477 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:34.115409 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vfxk\" (UniqueName: \"kubernetes.io/projected/a85e1f40-b852-4024-a1eb-0c5798b081e7-kube-api-access-6vfxk\") pod \"maas-default-gateway-openshift-default-66db998887-2jqqp\" (UID: \"a85e1f40-b852-4024-a1eb-0c5798b081e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp" Nov 03 21:56:34.115477 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:34.115436 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a85e1f40-b852-4024-a1eb-0c5798b081e7-istio-data\") pod \"maas-default-gateway-openshift-default-66db998887-2jqqp\" (UID: \"a85e1f40-b852-4024-a1eb-0c5798b081e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp" Nov 03 21:56:34.115719 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:34.115590 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a85e1f40-b852-4024-a1eb-0c5798b081e7-workload-socket\") pod \"maas-default-gateway-openshift-default-66db998887-2jqqp\" (UID: \"a85e1f40-b852-4024-a1eb-0c5798b081e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp" Nov 03 21:56:34.115719 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:34.115637 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a85e1f40-b852-4024-a1eb-0c5798b081e7-istio-token\") pod \"maas-default-gateway-openshift-default-66db998887-2jqqp\" (UID: \"a85e1f40-b852-4024-a1eb-0c5798b081e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp" Nov 03 21:56:34.115719 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:34.115679 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a85e1f40-b852-4024-a1eb-0c5798b081e7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-66db998887-2jqqp\" (UID: \"a85e1f40-b852-4024-a1eb-0c5798b081e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp" Nov 03 21:56:34.115914 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:34.115748 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a85e1f40-b852-4024-a1eb-0c5798b081e7-workload-certs\") pod \"maas-default-gateway-openshift-default-66db998887-2jqqp\" (UID: \"a85e1f40-b852-4024-a1eb-0c5798b081e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp" Nov 03 21:56:34.115914 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:34.115815 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a85e1f40-b852-4024-a1eb-0c5798b081e7-credential-socket\") pod \"maas-default-gateway-openshift-default-66db998887-2jqqp\" (UID: \"a85e1f40-b852-4024-a1eb-0c5798b081e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp" Nov 03 21:56:34.115914 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:34.115875 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a85e1f40-b852-4024-a1eb-0c5798b081e7-istio-data\") pod \"maas-default-gateway-openshift-default-66db998887-2jqqp\" (UID: \"a85e1f40-b852-4024-a1eb-0c5798b081e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp" Nov 03 21:56:34.116044 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:34.115965 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a85e1f40-b852-4024-a1eb-0c5798b081e7-workload-socket\") pod \"maas-default-gateway-openshift-default-66db998887-2jqqp\" (UID: \"a85e1f40-b852-4024-a1eb-0c5798b081e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp" Nov 03 21:56:34.116082 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:34.116051 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a85e1f40-b852-4024-a1eb-0c5798b081e7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-66db998887-2jqqp\" (UID: \"a85e1f40-b852-4024-a1eb-0c5798b081e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp" Nov 03 21:56:34.117707 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:34.117685 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a85e1f40-b852-4024-a1eb-0c5798b081e7-istio-envoy\") pod \"maas-default-gateway-openshift-default-66db998887-2jqqp\" (UID: \"a85e1f40-b852-4024-a1eb-0c5798b081e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp" Nov 03 21:56:34.117966 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:34.117949 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a85e1f40-b852-4024-a1eb-0c5798b081e7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-66db998887-2jqqp\" (UID: \"a85e1f40-b852-4024-a1eb-0c5798b081e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp" Nov 03 21:56:34.128486 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:34.128453 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a85e1f40-b852-4024-a1eb-0c5798b081e7-istio-token\") pod \"maas-default-gateway-openshift-default-66db998887-2jqqp\" (UID: \"a85e1f40-b852-4024-a1eb-0c5798b081e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp" Nov 03 21:56:34.128613 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:34.128564 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vfxk\" (UniqueName: \"kubernetes.io/projected/a85e1f40-b852-4024-a1eb-0c5798b081e7-kube-api-access-6vfxk\") pod \"maas-default-gateway-openshift-default-66db998887-2jqqp\" (UID: \"a85e1f40-b852-4024-a1eb-0c5798b081e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp" Nov 03 21:56:34.215032 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:34.214943 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp" Nov 03 21:56:34.492961 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:34.492888 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp"] Nov 03 21:56:34.496095 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:56:34.496069 6524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda85e1f40_b852_4024_a1eb_0c5798b081e7.slice/crio-232ca222523ac764f535dbba4633c69d65e0cc81d58d2545304deb1b871e1ce4 WatchSource:0}: Error finding container 232ca222523ac764f535dbba4633c69d65e0cc81d58d2545304deb1b871e1ce4: Status 404 returned error can't find the container with id 232ca222523ac764f535dbba4633c69d65e0cc81d58d2545304deb1b871e1ce4 Nov 03 21:56:35.207900 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:35.207867 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp" event={"ID":"a85e1f40-b852-4024-a1eb-0c5798b081e7","Type":"ContainerStarted","Data":"232ca222523ac764f535dbba4633c69d65e0cc81d58d2545304deb1b871e1ce4"} Nov 03 21:56:38.544231 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:38.544198 6524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"} Nov 03 21:56:38.544484 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:38.544272 6524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"} Nov 03 21:56:38.544484 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:38.544308 6524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"} Nov 03 21:56:39.227199 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:39.227162 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp" event={"ID":"a85e1f40-b852-4024-a1eb-0c5798b081e7","Type":"ContainerStarted","Data":"1897ed196fc4c517808f2886adec76d022c7ce7b655515e8b2994ce922dd6e91"} Nov 03 21:56:39.254347 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:39.254296 6524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp" podStartSLOduration=2.208892901 podStartE2EDuration="6.254280392s" totalImagesPullingTime="4.045387491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:56:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:56:39.252562544 +0000 UTC m=+644.604889361" watchObservedRunningTime="2026-11-03 21:56:39.254280392 +0000 UTC m=+644.606607196" Nov 03 21:56:40.215467 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:40.215418 6524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp" Nov 03 21:56:40.217047 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:40.217016 6524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-2jqqp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" start-of-body= Nov 03 21:56:40.217182 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:40.217082 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp" podUID="a85e1f40-b852-4024-a1eb-0c5798b081e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" Nov 03 21:56:41.216237 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:41.216195 6524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-2jqqp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" start-of-body= Nov 03 21:56:41.216721 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:41.216261 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp" podUID="a85e1f40-b852-4024-a1eb-0c5798b081e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" Nov 03 21:56:42.215712 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:42.215664 6524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-2jqqp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" start-of-body= Nov 03 21:56:42.215933 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:42.215730 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp" podUID="a85e1f40-b852-4024-a1eb-0c5798b081e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" Nov 03 21:56:43.215683 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:43.215643 6524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-2jqqp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" start-of-body= Nov 03 21:56:43.216098 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:43.215705 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp" podUID="a85e1f40-b852-4024-a1eb-0c5798b081e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" Nov 03 21:56:44.215870 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:44.215831 6524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-2jqqp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" start-of-body= Nov 03 21:56:44.216380 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:44.215894 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp" podUID="a85e1f40-b852-4024-a1eb-0c5798b081e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" Nov 03 21:56:45.216267 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:45.216224 6524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-2jqqp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" start-of-body= Nov 03 21:56:45.216686 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:45.216289 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp" podUID="a85e1f40-b852-4024-a1eb-0c5798b081e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" Nov 03 21:56:46.834349 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:46.834310 6524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp" Nov 03 21:56:46.834866 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:46.834748 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp" Nov 03 21:56:46.835883 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:46.835858 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp" Nov 03 21:56:48.922852 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:48.922814 6524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd"] Nov 03 21:56:48.926183 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:48.926116 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd" Nov 03 21:56:48.942191 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:48.942162 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd"] Nov 03 21:56:49.048957 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:49.048919 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09c7b2df-6149-4b8a-b8ed-c416fc097757-istio-podinfo\") pod \"maas-default-gateway-openshift-default-66db998887-knxdd\" (UID: \"09c7b2df-6149-4b8a-b8ed-c416fc097757\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd" Nov 03 21:56:49.048957 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:49.048964 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09c7b2df-6149-4b8a-b8ed-c416fc097757-workload-certs\") pod \"maas-default-gateway-openshift-default-66db998887-knxdd\" (UID: \"09c7b2df-6149-4b8a-b8ed-c416fc097757\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd" Nov 03 21:56:49.049232 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:49.048984 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09c7b2df-6149-4b8a-b8ed-c416fc097757-credential-socket\") pod \"maas-default-gateway-openshift-default-66db998887-knxdd\" (UID: \"09c7b2df-6149-4b8a-b8ed-c416fc097757\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd" Nov 03 21:56:49.049232 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:49.049108 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjvw6\" (UniqueName: \"kubernetes.io/projected/09c7b2df-6149-4b8a-b8ed-c416fc097757-kube-api-access-vjvw6\") pod \"maas-default-gateway-openshift-default-66db998887-knxdd\" (UID: \"09c7b2df-6149-4b8a-b8ed-c416fc097757\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd" Nov 03 21:56:49.049232 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:49.049198 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09c7b2df-6149-4b8a-b8ed-c416fc097757-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-66db998887-knxdd\" (UID: \"09c7b2df-6149-4b8a-b8ed-c416fc097757\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd" Nov 03 21:56:49.049344 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:49.049259 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09c7b2df-6149-4b8a-b8ed-c416fc097757-istio-data\") pod \"maas-default-gateway-openshift-default-66db998887-knxdd\" (UID: \"09c7b2df-6149-4b8a-b8ed-c416fc097757\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd" Nov 03 21:56:49.049344 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:49.049285 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09c7b2df-6149-4b8a-b8ed-c416fc097757-istio-token\") pod \"maas-default-gateway-openshift-default-66db998887-knxdd\" (UID: \"09c7b2df-6149-4b8a-b8ed-c416fc097757\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd" Nov 03 21:56:49.049426 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:49.049363 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09c7b2df-6149-4b8a-b8ed-c416fc097757-workload-socket\") pod \"maas-default-gateway-openshift-default-66db998887-knxdd\" (UID: \"09c7b2df-6149-4b8a-b8ed-c416fc097757\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd" Nov 03 21:56:49.049426 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:49.049396 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09c7b2df-6149-4b8a-b8ed-c416fc097757-istio-envoy\") pod \"maas-default-gateway-openshift-default-66db998887-knxdd\" (UID: \"09c7b2df-6149-4b8a-b8ed-c416fc097757\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd" Nov 03 21:56:49.150352 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:49.150309 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09c7b2df-6149-4b8a-b8ed-c416fc097757-istio-data\") pod \"maas-default-gateway-openshift-default-66db998887-knxdd\" (UID: \"09c7b2df-6149-4b8a-b8ed-c416fc097757\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd" Nov 03 21:56:49.150560 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:49.150359 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09c7b2df-6149-4b8a-b8ed-c416fc097757-istio-token\") pod \"maas-default-gateway-openshift-default-66db998887-knxdd\" (UID: \"09c7b2df-6149-4b8a-b8ed-c416fc097757\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd" Nov 03 21:56:49.150560 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:49.150389 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09c7b2df-6149-4b8a-b8ed-c416fc097757-workload-socket\") pod \"maas-default-gateway-openshift-default-66db998887-knxdd\" (UID: \"09c7b2df-6149-4b8a-b8ed-c416fc097757\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd" Nov 03 21:56:49.150560 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:49.150424 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09c7b2df-6149-4b8a-b8ed-c416fc097757-istio-envoy\") pod \"maas-default-gateway-openshift-default-66db998887-knxdd\" (UID: \"09c7b2df-6149-4b8a-b8ed-c416fc097757\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd" Nov 03 21:56:49.150560 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:49.150465 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09c7b2df-6149-4b8a-b8ed-c416fc097757-istio-podinfo\") pod \"maas-default-gateway-openshift-default-66db998887-knxdd\" (UID: \"09c7b2df-6149-4b8a-b8ed-c416fc097757\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd" Nov 03 21:56:49.150560 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:49.150503 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09c7b2df-6149-4b8a-b8ed-c416fc097757-workload-certs\") pod \"maas-default-gateway-openshift-default-66db998887-knxdd\" (UID: \"09c7b2df-6149-4b8a-b8ed-c416fc097757\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd" Nov 03 21:56:49.150560 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:49.150529 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09c7b2df-6149-4b8a-b8ed-c416fc097757-credential-socket\") pod \"maas-default-gateway-openshift-default-66db998887-knxdd\" (UID: \"09c7b2df-6149-4b8a-b8ed-c416fc097757\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd" Nov 03 21:56:49.150560 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:49.150558 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjvw6\" (UniqueName: \"kubernetes.io/projected/09c7b2df-6149-4b8a-b8ed-c416fc097757-kube-api-access-vjvw6\") pod \"maas-default-gateway-openshift-default-66db998887-knxdd\" (UID: \"09c7b2df-6149-4b8a-b8ed-c416fc097757\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd" Nov 03 21:56:49.150923 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:49.150591 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09c7b2df-6149-4b8a-b8ed-c416fc097757-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-66db998887-knxdd\" (UID: \"09c7b2df-6149-4b8a-b8ed-c416fc097757\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd" Nov 03 21:56:49.150923 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:49.150795 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09c7b2df-6149-4b8a-b8ed-c416fc097757-istio-data\") pod \"maas-default-gateway-openshift-default-66db998887-knxdd\" (UID: \"09c7b2df-6149-4b8a-b8ed-c416fc097757\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd" Nov 03 21:56:49.150923 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:49.150849 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09c7b2df-6149-4b8a-b8ed-c416fc097757-workload-socket\") pod \"maas-default-gateway-openshift-default-66db998887-knxdd\" (UID: \"09c7b2df-6149-4b8a-b8ed-c416fc097757\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd" Nov 03 21:56:49.150923 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:49.150890 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09c7b2df-6149-4b8a-b8ed-c416fc097757-workload-certs\") pod \"maas-default-gateway-openshift-default-66db998887-knxdd\" (UID: \"09c7b2df-6149-4b8a-b8ed-c416fc097757\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd" Nov 03 21:56:49.151164 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:49.151111 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09c7b2df-6149-4b8a-b8ed-c416fc097757-credential-socket\") pod \"maas-default-gateway-openshift-default-66db998887-knxdd\" (UID: \"09c7b2df-6149-4b8a-b8ed-c416fc097757\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd" Nov 03 21:56:49.151419 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:49.151396 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09c7b2df-6149-4b8a-b8ed-c416fc097757-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-66db998887-knxdd\" (UID: \"09c7b2df-6149-4b8a-b8ed-c416fc097757\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd" Nov 03 21:56:49.152816 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:49.152795 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09c7b2df-6149-4b8a-b8ed-c416fc097757-istio-envoy\") pod \"maas-default-gateway-openshift-default-66db998887-knxdd\" (UID: \"09c7b2df-6149-4b8a-b8ed-c416fc097757\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd" Nov 03 21:56:49.153135 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:49.153105 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09c7b2df-6149-4b8a-b8ed-c416fc097757-istio-podinfo\") pod \"maas-default-gateway-openshift-default-66db998887-knxdd\" (UID: \"09c7b2df-6149-4b8a-b8ed-c416fc097757\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd" Nov 03 21:56:49.162983 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:49.162954 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09c7b2df-6149-4b8a-b8ed-c416fc097757-istio-token\") pod \"maas-default-gateway-openshift-default-66db998887-knxdd\" (UID: \"09c7b2df-6149-4b8a-b8ed-c416fc097757\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd" Nov 03 21:56:49.163117 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:49.163098 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjvw6\" (UniqueName: \"kubernetes.io/projected/09c7b2df-6149-4b8a-b8ed-c416fc097757-kube-api-access-vjvw6\") pod \"maas-default-gateway-openshift-default-66db998887-knxdd\" (UID: \"09c7b2df-6149-4b8a-b8ed-c416fc097757\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd" Nov 03 21:56:49.240268 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:49.240167 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd" Nov 03 21:56:49.519299 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:49.519218 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd"] Nov 03 21:56:49.522844 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:56:49.522807 6524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09c7b2df_6149_4b8a_b8ed_c416fc097757.slice/crio-5e82940da266d8221eff37090f21599f82c8208fa8814650dea6b5772669de38 WatchSource:0}: Error finding container 5e82940da266d8221eff37090f21599f82c8208fa8814650dea6b5772669de38: Status 404 returned error can't find the container with id 5e82940da266d8221eff37090f21599f82c8208fa8814650dea6b5772669de38 Nov 03 21:56:49.525621 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:49.525590 6524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"} Nov 03 21:56:49.525727 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:49.525688 6524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"} Nov 03 21:56:49.525790 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:49.525738 6524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"} Nov 03 21:56:50.273016 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:50.272982 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd" event={"ID":"09c7b2df-6149-4b8a-b8ed-c416fc097757","Type":"ContainerStarted","Data":"3942afcd2bc51b0362eecd606705213900ca9cba3f58e50cd0dc27ccb7fe5ff9"} Nov 03 21:56:50.273443 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:50.273034 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd" event={"ID":"09c7b2df-6149-4b8a-b8ed-c416fc097757","Type":"ContainerStarted","Data":"5e82940da266d8221eff37090f21599f82c8208fa8814650dea6b5772669de38"} Nov 03 21:56:51.241188 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:51.241144 6524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd" Nov 03 21:56:51.242524 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:51.242504 6524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-knxdd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body= Nov 03 21:56:51.242594 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:51.242549 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd" podUID="09c7b2df-6149-4b8a-b8ed-c416fc097757" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" Nov 03 21:56:52.240587 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:52.240549 6524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-knxdd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body= Nov 03 21:56:52.240997 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:52.240606 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd" podUID="09c7b2df-6149-4b8a-b8ed-c416fc097757" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" Nov 03 21:56:53.240904 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:53.240867 6524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-knxdd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body= Nov 03 21:56:53.241346 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:53.240939 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd" podUID="09c7b2df-6149-4b8a-b8ed-c416fc097757" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" Nov 03 21:56:54.241490 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:54.241444 6524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-knxdd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body= Nov 03 21:56:54.241992 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:54.241501 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd" podUID="09c7b2df-6149-4b8a-b8ed-c416fc097757" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" Nov 03 21:56:55.240929 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:55.240890 6524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-knxdd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body= Nov 03 21:56:55.241111 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:55.240957 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd" podUID="09c7b2df-6149-4b8a-b8ed-c416fc097757" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" Nov 03 21:56:56.240918 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:56.240881 6524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-knxdd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body= Nov 03 21:56:56.241429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:56.240935 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd" podUID="09c7b2df-6149-4b8a-b8ed-c416fc097757" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" Nov 03 21:56:57.770770 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:57.770736 6524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd" Nov 03 21:56:57.771275 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:57.771220 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd" Nov 03 21:56:57.772217 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:57.772196 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd" Nov 03 21:56:57.804531 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:56:57.804473 6524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd" podStartSLOduration=9.804458912 podStartE2EDuration="9.804458912s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:56:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:56:50.307848434 +0000 UTC m=+655.660175239" watchObservedRunningTime="2026-11-03 21:56:57.804458912 +0000 UTC m=+663.156785709" Nov 03 21:56:59.706496 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:56:59.706454 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 21:57:03.937649 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:03.937613 6524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx"] Nov 03 21:57:03.944642 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:03.944618 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx" Nov 03 21:57:03.955062 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:03.955033 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx"] Nov 03 21:57:04.004672 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:04.004635 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3379b31a-b827-4511-b8c8-326121b22989-workload-socket\") pod \"maas-default-gateway-openshift-default-66db998887-6czqx\" (UID: \"3379b31a-b827-4511-b8c8-326121b22989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx" Nov 03 21:57:04.004672 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:04.004672 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knm2z\" (UniqueName: \"kubernetes.io/projected/3379b31a-b827-4511-b8c8-326121b22989-kube-api-access-knm2z\") pod \"maas-default-gateway-openshift-default-66db998887-6czqx\" (UID: \"3379b31a-b827-4511-b8c8-326121b22989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx" Nov 03 21:57:04.004886 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:04.004713 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3379b31a-b827-4511-b8c8-326121b22989-istio-token\") pod \"maas-default-gateway-openshift-default-66db998887-6czqx\" (UID: \"3379b31a-b827-4511-b8c8-326121b22989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx" Nov 03 21:57:04.004886 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:04.004795 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3379b31a-b827-4511-b8c8-326121b22989-workload-certs\") pod \"maas-default-gateway-openshift-default-66db998887-6czqx\" (UID: \"3379b31a-b827-4511-b8c8-326121b22989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx" Nov 03 21:57:04.004886 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:04.004843 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3379b31a-b827-4511-b8c8-326121b22989-istio-data\") pod \"maas-default-gateway-openshift-default-66db998887-6czqx\" (UID: \"3379b31a-b827-4511-b8c8-326121b22989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx" Nov 03 21:57:04.004886 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:04.004860 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3379b31a-b827-4511-b8c8-326121b22989-istio-podinfo\") pod \"maas-default-gateway-openshift-default-66db998887-6czqx\" (UID: \"3379b31a-b827-4511-b8c8-326121b22989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx" Nov 03 21:57:04.005013 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:04.004923 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3379b31a-b827-4511-b8c8-326121b22989-istio-envoy\") pod \"maas-default-gateway-openshift-default-66db998887-6czqx\" (UID: \"3379b31a-b827-4511-b8c8-326121b22989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx" Nov 03 21:57:04.005013 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:04.004950 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3379b31a-b827-4511-b8c8-326121b22989-credential-socket\") pod \"maas-default-gateway-openshift-default-66db998887-6czqx\" (UID: \"3379b31a-b827-4511-b8c8-326121b22989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx" Nov 03 21:57:04.005013 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:04.004980 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3379b31a-b827-4511-b8c8-326121b22989-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-66db998887-6czqx\" (UID: \"3379b31a-b827-4511-b8c8-326121b22989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx" Nov 03 21:57:04.105553 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:04.105512 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3379b31a-b827-4511-b8c8-326121b22989-istio-data\") pod \"maas-default-gateway-openshift-default-66db998887-6czqx\" (UID: \"3379b31a-b827-4511-b8c8-326121b22989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx" Nov 03 21:57:04.105553 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:04.105548 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3379b31a-b827-4511-b8c8-326121b22989-istio-podinfo\") pod \"maas-default-gateway-openshift-default-66db998887-6czqx\" (UID: \"3379b31a-b827-4511-b8c8-326121b22989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx" Nov 03 21:57:04.105763 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:04.105569 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3379b31a-b827-4511-b8c8-326121b22989-istio-envoy\") pod \"maas-default-gateway-openshift-default-66db998887-6czqx\" (UID: \"3379b31a-b827-4511-b8c8-326121b22989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx" Nov 03 21:57:04.105763 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:04.105589 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3379b31a-b827-4511-b8c8-326121b22989-credential-socket\") pod \"maas-default-gateway-openshift-default-66db998887-6czqx\" (UID: \"3379b31a-b827-4511-b8c8-326121b22989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx" Nov 03 21:57:04.105763 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:04.105616 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3379b31a-b827-4511-b8c8-326121b22989-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-66db998887-6czqx\" (UID: \"3379b31a-b827-4511-b8c8-326121b22989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx" Nov 03 21:57:04.105763 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:04.105736 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3379b31a-b827-4511-b8c8-326121b22989-workload-socket\") pod \"maas-default-gateway-openshift-default-66db998887-6czqx\" (UID: \"3379b31a-b827-4511-b8c8-326121b22989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx" Nov 03 21:57:04.105949 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:04.105765 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knm2z\" (UniqueName: \"kubernetes.io/projected/3379b31a-b827-4511-b8c8-326121b22989-kube-api-access-knm2z\") pod \"maas-default-gateway-openshift-default-66db998887-6czqx\" (UID: \"3379b31a-b827-4511-b8c8-326121b22989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx" Nov 03 21:57:04.105949 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:04.105812 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3379b31a-b827-4511-b8c8-326121b22989-istio-token\") pod \"maas-default-gateway-openshift-default-66db998887-6czqx\" (UID: \"3379b31a-b827-4511-b8c8-326121b22989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx" Nov 03 21:57:04.105949 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:04.105864 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3379b31a-b827-4511-b8c8-326121b22989-workload-certs\") pod \"maas-default-gateway-openshift-default-66db998887-6czqx\" (UID: \"3379b31a-b827-4511-b8c8-326121b22989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx" Nov 03 21:57:04.106090 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:04.106016 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3379b31a-b827-4511-b8c8-326121b22989-credential-socket\") pod \"maas-default-gateway-openshift-default-66db998887-6czqx\" (UID: \"3379b31a-b827-4511-b8c8-326121b22989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx" Nov 03 21:57:04.106090 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:04.106033 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3379b31a-b827-4511-b8c8-326121b22989-istio-data\") pod \"maas-default-gateway-openshift-default-66db998887-6czqx\" (UID: \"3379b31a-b827-4511-b8c8-326121b22989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx" Nov 03 21:57:04.106223 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:04.106166 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3379b31a-b827-4511-b8c8-326121b22989-workload-socket\") pod \"maas-default-gateway-openshift-default-66db998887-6czqx\" (UID: \"3379b31a-b827-4511-b8c8-326121b22989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx" Nov 03 21:57:04.106291 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:04.106216 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3379b31a-b827-4511-b8c8-326121b22989-workload-certs\") pod \"maas-default-gateway-openshift-default-66db998887-6czqx\" (UID: \"3379b31a-b827-4511-b8c8-326121b22989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx" Nov 03 21:57:04.106517 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:04.106497 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3379b31a-b827-4511-b8c8-326121b22989-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-66db998887-6czqx\" (UID: \"3379b31a-b827-4511-b8c8-326121b22989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx" Nov 03 21:57:04.108071 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:04.108047 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3379b31a-b827-4511-b8c8-326121b22989-istio-envoy\") pod \"maas-default-gateway-openshift-default-66db998887-6czqx\" (UID: \"3379b31a-b827-4511-b8c8-326121b22989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx" Nov 03 21:57:04.108195 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:04.108099 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3379b31a-b827-4511-b8c8-326121b22989-istio-podinfo\") pod \"maas-default-gateway-openshift-default-66db998887-6czqx\" (UID: \"3379b31a-b827-4511-b8c8-326121b22989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx" Nov 03 21:57:04.118654 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:04.118627 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3379b31a-b827-4511-b8c8-326121b22989-istio-token\") pod \"maas-default-gateway-openshift-default-66db998887-6czqx\" (UID: \"3379b31a-b827-4511-b8c8-326121b22989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx" Nov 03 21:57:04.118790 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:04.118772 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-knm2z\" (UniqueName: \"kubernetes.io/projected/3379b31a-b827-4511-b8c8-326121b22989-kube-api-access-knm2z\") pod \"maas-default-gateway-openshift-default-66db998887-6czqx\" (UID: \"3379b31a-b827-4511-b8c8-326121b22989\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx" Nov 03 21:57:04.258367 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:04.258253 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx" Nov 03 21:57:04.535363 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:04.535289 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx"] Nov 03 21:57:04.538454 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:57:04.538425 6524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3379b31a_b827_4511_b8c8_326121b22989.slice/crio-d33a1b7287f86fad9a8562499c7d9f7497bb792ffec5937ba3446d5943fd0c99 WatchSource:0}: Error finding container d33a1b7287f86fad9a8562499c7d9f7497bb792ffec5937ba3446d5943fd0c99: Status 404 returned error can't find the container with id d33a1b7287f86fad9a8562499c7d9f7497bb792ffec5937ba3446d5943fd0c99 Nov 03 21:57:04.540462 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:04.540428 6524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"} Nov 03 21:57:04.540554 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:04.540508 6524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"} Nov 03 21:57:04.540554 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:04.540543 6524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"} Nov 03 21:57:05.344795 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:05.344758 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx" event={"ID":"3379b31a-b827-4511-b8c8-326121b22989","Type":"ContainerStarted","Data":"c8293af59bf48c20c1109977e6e67a405be877054b834b952e3f3aca4e2166ec"} Nov 03 21:57:05.345212 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:05.344823 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx" event={"ID":"3379b31a-b827-4511-b8c8-326121b22989","Type":"ContainerStarted","Data":"d33a1b7287f86fad9a8562499c7d9f7497bb792ffec5937ba3446d5943fd0c99"} Nov 03 21:57:05.370762 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:05.370712 6524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx" podStartSLOduration=2.370695391 podStartE2EDuration="2.370695391s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:57:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:57:05.369377353 +0000 UTC m=+670.721704158" watchObservedRunningTime="2026-11-03 21:57:05.370695391 +0000 UTC m=+670.723022193" Nov 03 21:57:06.258711 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:06.258678 6524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx" Nov 03 21:57:06.260153 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:06.260113 6524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-6czqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body= Nov 03 21:57:06.260232 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:06.260209 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx" podUID="3379b31a-b827-4511-b8c8-326121b22989" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" Nov 03 21:57:07.259326 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:07.259281 6524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-6czqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body= Nov 03 21:57:07.259749 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:07.259343 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx" podUID="3379b31a-b827-4511-b8c8-326121b22989" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" Nov 03 21:57:08.259279 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:08.259236 6524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-6czqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body= Nov 03 21:57:08.259478 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:08.259308 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx" podUID="3379b31a-b827-4511-b8c8-326121b22989" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" Nov 03 21:57:09.259327 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:09.259292 6524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-6czqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body= Nov 03 21:57:09.259517 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:09.259348 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx" podUID="3379b31a-b827-4511-b8c8-326121b22989" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" Nov 03 21:57:10.259209 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:10.259167 6524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-6czqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body= Nov 03 21:57:10.259405 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:10.259227 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx" podUID="3379b31a-b827-4511-b8c8-326121b22989" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" Nov 03 21:57:11.258839 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:11.258797 6524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-6czqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body= Nov 03 21:57:11.259267 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:11.258869 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx" podUID="3379b31a-b827-4511-b8c8-326121b22989" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" Nov 03 21:57:12.747910 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:12.747868 6524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx" Nov 03 21:57:12.748386 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:12.748222 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx" Nov 03 21:57:12.749088 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:12.749066 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx" Nov 03 21:57:21.900193 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:21.900152 6524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j"] Nov 03 21:57:21.903951 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:21.903928 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" Nov 03 21:57:21.921377 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:21.921345 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j"] Nov 03 21:57:21.972708 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:21.972644 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82c4e70e-4765-48d3-80d3-8e0311abfda7-istio-envoy\") pod \"e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j\" (UID: \"82c4e70e-4765-48d3-80d3-8e0311abfda7\") " pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" Nov 03 21:57:21.972917 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:21.972787 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82c4e70e-4765-48d3-80d3-8e0311abfda7-istio-data\") pod \"e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j\" (UID: \"82c4e70e-4765-48d3-80d3-8e0311abfda7\") " pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" Nov 03 21:57:21.972917 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:21.972826 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82c4e70e-4765-48d3-80d3-8e0311abfda7-workload-certs\") pod \"e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j\" (UID: \"82c4e70e-4765-48d3-80d3-8e0311abfda7\") " pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" Nov 03 21:57:21.972917 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:21.972846 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbh4z\" (UniqueName: \"kubernetes.io/projected/82c4e70e-4765-48d3-80d3-8e0311abfda7-kube-api-access-hbh4z\") pod \"e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j\" (UID: \"82c4e70e-4765-48d3-80d3-8e0311abfda7\") " pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" Nov 03 21:57:21.973083 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:21.972948 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82c4e70e-4765-48d3-80d3-8e0311abfda7-workload-socket\") pod \"e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j\" (UID: \"82c4e70e-4765-48d3-80d3-8e0311abfda7\") " pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" Nov 03 21:57:21.973083 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:21.973004 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82c4e70e-4765-48d3-80d3-8e0311abfda7-istio-token\") pod \"e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j\" (UID: \"82c4e70e-4765-48d3-80d3-8e0311abfda7\") " pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" Nov 03 21:57:21.973083 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:21.973025 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82c4e70e-4765-48d3-80d3-8e0311abfda7-credential-socket\") pod \"e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j\" (UID: \"82c4e70e-4765-48d3-80d3-8e0311abfda7\") " pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" Nov 03 21:57:21.973083 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:21.973044 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82c4e70e-4765-48d3-80d3-8e0311abfda7-istio-podinfo\") pod \"e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j\" (UID: \"82c4e70e-4765-48d3-80d3-8e0311abfda7\") " pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" Nov 03 21:57:21.973312 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:21.973104 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82c4e70e-4765-48d3-80d3-8e0311abfda7-istiod-ca-cert\") pod \"e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j\" (UID: \"82c4e70e-4765-48d3-80d3-8e0311abfda7\") " pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" Nov 03 21:57:21.982305 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:21.981847 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-c0caef-openshift-default-dockercfg-4vqx5\"" Nov 03 21:57:22.073875 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:22.073823 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82c4e70e-4765-48d3-80d3-8e0311abfda7-istio-data\") pod \"e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j\" (UID: \"82c4e70e-4765-48d3-80d3-8e0311abfda7\") " pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" Nov 03 21:57:22.073875 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:22.073870 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82c4e70e-4765-48d3-80d3-8e0311abfda7-workload-certs\") pod \"e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j\" (UID: \"82c4e70e-4765-48d3-80d3-8e0311abfda7\") " pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" Nov 03 21:57:22.074155 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:22.073891 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbh4z\" (UniqueName: \"kubernetes.io/projected/82c4e70e-4765-48d3-80d3-8e0311abfda7-kube-api-access-hbh4z\") pod \"e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j\" (UID: \"82c4e70e-4765-48d3-80d3-8e0311abfda7\") " pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" Nov 03 21:57:22.074155 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:22.073935 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82c4e70e-4765-48d3-80d3-8e0311abfda7-workload-socket\") pod \"e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j\" (UID: \"82c4e70e-4765-48d3-80d3-8e0311abfda7\") " pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" Nov 03 21:57:22.074155 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:22.074003 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82c4e70e-4765-48d3-80d3-8e0311abfda7-istio-token\") pod \"e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j\" (UID: \"82c4e70e-4765-48d3-80d3-8e0311abfda7\") " pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" Nov 03 21:57:22.074155 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:22.074032 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82c4e70e-4765-48d3-80d3-8e0311abfda7-credential-socket\") pod \"e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j\" (UID: \"82c4e70e-4765-48d3-80d3-8e0311abfda7\") " pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" Nov 03 21:57:22.074155 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:22.074060 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82c4e70e-4765-48d3-80d3-8e0311abfda7-istio-podinfo\") pod \"e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j\" (UID: \"82c4e70e-4765-48d3-80d3-8e0311abfda7\") " pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" Nov 03 21:57:22.074155 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:22.074090 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82c4e70e-4765-48d3-80d3-8e0311abfda7-istiod-ca-cert\") pod \"e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j\" (UID: \"82c4e70e-4765-48d3-80d3-8e0311abfda7\") " pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" Nov 03 21:57:22.074453 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:22.074194 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82c4e70e-4765-48d3-80d3-8e0311abfda7-istio-envoy\") pod \"e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j\" (UID: \"82c4e70e-4765-48d3-80d3-8e0311abfda7\") " pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" Nov 03 21:57:22.074453 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:22.074346 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82c4e70e-4765-48d3-80d3-8e0311abfda7-istio-data\") pod \"e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j\" (UID: \"82c4e70e-4765-48d3-80d3-8e0311abfda7\") " pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" Nov 03 21:57:22.074453 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:22.074421 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82c4e70e-4765-48d3-80d3-8e0311abfda7-workload-certs\") pod \"e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j\" (UID: \"82c4e70e-4765-48d3-80d3-8e0311abfda7\") " pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" Nov 03 21:57:22.074703 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:22.074681 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82c4e70e-4765-48d3-80d3-8e0311abfda7-credential-socket\") pod \"e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j\" (UID: \"82c4e70e-4765-48d3-80d3-8e0311abfda7\") " pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" Nov 03 21:57:22.075068 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:22.075039 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82c4e70e-4765-48d3-80d3-8e0311abfda7-workload-socket\") pod \"e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j\" (UID: \"82c4e70e-4765-48d3-80d3-8e0311abfda7\") " pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" Nov 03 21:57:22.075262 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:22.075239 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82c4e70e-4765-48d3-80d3-8e0311abfda7-istiod-ca-cert\") pod \"e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j\" (UID: \"82c4e70e-4765-48d3-80d3-8e0311abfda7\") " pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" Nov 03 21:57:22.076618 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:22.076594 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82c4e70e-4765-48d3-80d3-8e0311abfda7-istio-envoy\") pod \"e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j\" (UID: \"82c4e70e-4765-48d3-80d3-8e0311abfda7\") " pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" Nov 03 21:57:22.076945 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:22.076919 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82c4e70e-4765-48d3-80d3-8e0311abfda7-istio-podinfo\") pod \"e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j\" (UID: \"82c4e70e-4765-48d3-80d3-8e0311abfda7\") " pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" Nov 03 21:57:22.090486 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:22.088363 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbh4z\" (UniqueName: \"kubernetes.io/projected/82c4e70e-4765-48d3-80d3-8e0311abfda7-kube-api-access-hbh4z\") pod \"e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j\" (UID: \"82c4e70e-4765-48d3-80d3-8e0311abfda7\") " pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" Nov 03 21:57:22.091168 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:22.091089 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82c4e70e-4765-48d3-80d3-8e0311abfda7-istio-token\") pod \"e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j\" (UID: \"82c4e70e-4765-48d3-80d3-8e0311abfda7\") " pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" Nov 03 21:57:22.221833 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:22.221730 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" Nov 03 21:57:22.507550 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:22.507469 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j"] Nov 03 21:57:22.510744 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:57:22.510706 6524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82c4e70e_4765_48d3_80d3_8e0311abfda7.slice/crio-89b0495e70faf4ac937912ee7e9b5689ff0094387706e96c0098c72e96baf9eb WatchSource:0}: Error finding container 89b0495e70faf4ac937912ee7e9b5689ff0094387706e96c0098c72e96baf9eb: Status 404 returned error can't find the container with id 89b0495e70faf4ac937912ee7e9b5689ff0094387706e96c0098c72e96baf9eb Nov 03 21:57:22.512979 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:22.512947 6524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"} Nov 03 21:57:22.513054 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:22.513020 6524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"} Nov 03 21:57:22.513054 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:22.513048 6524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"} Nov 03 21:57:23.429623 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:23.429586 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" event={"ID":"82c4e70e-4765-48d3-80d3-8e0311abfda7","Type":"ContainerStarted","Data":"c8e6e032a42781b491f35490e3e79e129f0f8aa6b69418c47760b39ef261dec2"} Nov 03 21:57:23.430057 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:23.429649 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" event={"ID":"82c4e70e-4765-48d3-80d3-8e0311abfda7","Type":"ContainerStarted","Data":"89b0495e70faf4ac937912ee7e9b5689ff0094387706e96c0098c72e96baf9eb"} Nov 03 21:57:23.458196 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:23.458140 6524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" podStartSLOduration=2.458110981 podStartE2EDuration="2.458110981s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:57:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:57:23.4576936 +0000 UTC m=+688.810020405" watchObservedRunningTime="2026-11-03 21:57:23.458110981 +0000 UTC m=+688.810437784" Nov 03 21:57:24.222647 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:24.222597 6524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" Nov 03 21:57:24.229118 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:24.229088 6524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" Nov 03 21:57:24.434180 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:24.434148 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" Nov 03 21:57:24.435329 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:24.435309 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" Nov 03 21:57:29.709593 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:57:29.709544 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 21:57:30.708362 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:30.708324 6524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-8af3f3-7c9fcbb455-25425"] Nov 03 21:57:30.711452 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:30.711429 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-8af3f3-7c9fcbb455-25425" Nov 03 21:57:30.725798 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:30.725775 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-8af3f3-7c9fcbb455-25425"] Nov 03 21:57:30.726750 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:30.726723 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w6-8af3f3\"" Nov 03 21:57:30.761295 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:30.761258 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqcb9\" (UniqueName: \"kubernetes.io/projected/2e7d9548-49de-403f-8d4e-14eb05b8b338-kube-api-access-mqcb9\") pod \"payload-pre-processing-e2e-worker-w6-8af3f3-7c9fcbb455-25425\" (UID: \"2e7d9548-49de-403f-8d4e-14eb05b8b338\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-8af3f3-7c9fcbb455-25425" Nov 03 21:57:30.761481 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:30.761323 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e7d9548-49de-403f-8d4e-14eb05b8b338-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-8af3f3-7c9fcbb455-25425\" (UID: \"2e7d9548-49de-403f-8d4e-14eb05b8b338\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-8af3f3-7c9fcbb455-25425" Nov 03 21:57:30.767213 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:30.767180 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w6-8af3f3-dockercfg-d9bnt\"" Nov 03 21:57:30.776925 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:30.776899 6524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-8af3f3-655f44685b-wh9n2"] Nov 03 21:57:30.780330 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:30.780314 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-8af3f3-655f44685b-wh9n2" Nov 03 21:57:30.789464 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:30.789437 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-8af3f3-655f44685b-wh9n2"] Nov 03 21:57:30.862348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:30.862309 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xln6\" (UniqueName: \"kubernetes.io/projected/171f367e-e35d-4444-86b4-2842b49424fb-kube-api-access-8xln6\") pod \"payload-processing-e2e-worker-w6-8af3f3-655f44685b-wh9n2\" (UID: \"171f367e-e35d-4444-86b4-2842b49424fb\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-8af3f3-655f44685b-wh9n2" Nov 03 21:57:30.862561 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:30.862375 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqcb9\" (UniqueName: \"kubernetes.io/projected/2e7d9548-49de-403f-8d4e-14eb05b8b338-kube-api-access-mqcb9\") pod \"payload-pre-processing-e2e-worker-w6-8af3f3-7c9fcbb455-25425\" (UID: \"2e7d9548-49de-403f-8d4e-14eb05b8b338\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-8af3f3-7c9fcbb455-25425" Nov 03 21:57:30.862561 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:30.862468 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e7d9548-49de-403f-8d4e-14eb05b8b338-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-8af3f3-7c9fcbb455-25425\" (UID: \"2e7d9548-49de-403f-8d4e-14eb05b8b338\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-8af3f3-7c9fcbb455-25425" Nov 03 21:57:30.862561 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:30.862541 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/171f367e-e35d-4444-86b4-2842b49424fb-plugins-config-volume\") pod \"payload-processing-e2e-worker-w6-8af3f3-655f44685b-wh9n2\" (UID: \"171f367e-e35d-4444-86b4-2842b49424fb\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-8af3f3-655f44685b-wh9n2" Nov 03 21:57:30.863262 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:30.863243 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e7d9548-49de-403f-8d4e-14eb05b8b338-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-8af3f3-7c9fcbb455-25425\" (UID: \"2e7d9548-49de-403f-8d4e-14eb05b8b338\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-8af3f3-7c9fcbb455-25425" Nov 03 21:57:30.876868 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:30.876842 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqcb9\" (UniqueName: \"kubernetes.io/projected/2e7d9548-49de-403f-8d4e-14eb05b8b338-kube-api-access-mqcb9\") pod \"payload-pre-processing-e2e-worker-w6-8af3f3-7c9fcbb455-25425\" (UID: \"2e7d9548-49de-403f-8d4e-14eb05b8b338\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-8af3f3-7c9fcbb455-25425" Nov 03 21:57:30.963256 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:30.963163 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/171f367e-e35d-4444-86b4-2842b49424fb-plugins-config-volume\") pod \"payload-processing-e2e-worker-w6-8af3f3-655f44685b-wh9n2\" (UID: \"171f367e-e35d-4444-86b4-2842b49424fb\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-8af3f3-655f44685b-wh9n2" Nov 03 21:57:30.963256 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:30.963222 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xln6\" (UniqueName: \"kubernetes.io/projected/171f367e-e35d-4444-86b4-2842b49424fb-kube-api-access-8xln6\") pod \"payload-processing-e2e-worker-w6-8af3f3-655f44685b-wh9n2\" (UID: \"171f367e-e35d-4444-86b4-2842b49424fb\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-8af3f3-655f44685b-wh9n2" Nov 03 21:57:30.963925 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:30.963899 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/171f367e-e35d-4444-86b4-2842b49424fb-plugins-config-volume\") pod \"payload-processing-e2e-worker-w6-8af3f3-655f44685b-wh9n2\" (UID: \"171f367e-e35d-4444-86b4-2842b49424fb\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-8af3f3-655f44685b-wh9n2" Nov 03 21:57:30.976365 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:30.976322 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xln6\" (UniqueName: \"kubernetes.io/projected/171f367e-e35d-4444-86b4-2842b49424fb-kube-api-access-8xln6\") pod \"payload-processing-e2e-worker-w6-8af3f3-655f44685b-wh9n2\" (UID: \"171f367e-e35d-4444-86b4-2842b49424fb\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-8af3f3-655f44685b-wh9n2" Nov 03 21:57:31.025190 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:31.025159 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-8af3f3-7c9fcbb455-25425" Nov 03 21:57:31.093875 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:31.093565 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-8af3f3-655f44685b-wh9n2" Nov 03 21:57:31.304373 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:31.304345 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-8af3f3-7c9fcbb455-25425"] Nov 03 21:57:31.306877 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:57:31.306853 6524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e7d9548_49de_403f_8d4e_14eb05b8b338.slice/crio-edbf33d8f2044e7e76dff44ad7749c582d4ce162604c2f8c692a8888d5bcfe34 WatchSource:0}: Error finding container edbf33d8f2044e7e76dff44ad7749c582d4ce162604c2f8c692a8888d5bcfe34: Status 404 returned error can't find the container with id edbf33d8f2044e7e76dff44ad7749c582d4ce162604c2f8c692a8888d5bcfe34 Nov 03 21:57:31.397595 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:31.397559 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-8af3f3-655f44685b-wh9n2"] Nov 03 21:57:31.400749 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:57:31.400727 6524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod171f367e_e35d_4444_86b4_2842b49424fb.slice/crio-2ebde2cb2714b84fdc2a010a3bf3831a7361f503d6f6521552b2d723e6b5fb32 WatchSource:0}: Error finding container 2ebde2cb2714b84fdc2a010a3bf3831a7361f503d6f6521552b2d723e6b5fb32: Status 404 returned error can't find the container with id 2ebde2cb2714b84fdc2a010a3bf3831a7361f503d6f6521552b2d723e6b5fb32 Nov 03 21:57:31.469480 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:31.469442 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-8af3f3-7c9fcbb455-25425" event={"ID":"2e7d9548-49de-403f-8d4e-14eb05b8b338","Type":"ContainerStarted","Data":"edbf33d8f2044e7e76dff44ad7749c582d4ce162604c2f8c692a8888d5bcfe34"} Nov 03 21:57:31.470671 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:31.470644 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-8af3f3-655f44685b-wh9n2" event={"ID":"171f367e-e35d-4444-86b4-2842b49424fb","Type":"ContainerStarted","Data":"2ebde2cb2714b84fdc2a010a3bf3831a7361f503d6f6521552b2d723e6b5fb32"} Nov 03 21:57:34.488889 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:34.488854 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-8af3f3-655f44685b-wh9n2" event={"ID":"171f367e-e35d-4444-86b4-2842b49424fb","Type":"ContainerStarted","Data":"d977ae9bcb79749def02d4d045471895c08c7a6817720f02af879e2121d5970a"} Nov 03 21:57:34.489395 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:34.488937 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w6-8af3f3-655f44685b-wh9n2" Nov 03 21:57:34.490426 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:34.490405 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-8af3f3-7c9fcbb455-25425" event={"ID":"2e7d9548-49de-403f-8d4e-14eb05b8b338","Type":"ContainerStarted","Data":"7dfa6ab0c2b15fb9851ea22605e8a6c45b019d97b5f25490e6834e84a74ea5b9"} Nov 03 21:57:34.490527 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:34.490514 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-8af3f3-7c9fcbb455-25425" Nov 03 21:57:34.510115 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:34.510064 6524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w6-8af3f3-655f44685b-wh9n2" podStartSLOduration=1.980412152 podStartE2EDuration="4.510049921s" totalImagesPullingTime="2.529637769s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:57:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:57:34.509115344 +0000 UTC m=+699.861442159" watchObservedRunningTime="2026-11-03 21:57:34.510049921 +0000 UTC m=+699.862376769" Nov 03 21:57:34.532463 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:34.532403 6524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-8af3f3-7c9fcbb455-25425" podStartSLOduration=1.94283734 podStartE2EDuration="4.532383978s" totalImagesPullingTime="2.589546638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:57:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:57:34.531725885 +0000 UTC m=+699.884052690" watchObservedRunningTime="2026-11-03 21:57:34.532383978 +0000 UTC m=+699.884710781" Nov 03 21:57:34.949637 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:34.949597 6524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-8af3f3-8bcd46f45-qbvvd"] Nov 03 21:57:34.953850 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:34.953820 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-8af3f3-8bcd46f45-qbvvd" Nov 03 21:57:34.964540 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:34.963397 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w6-8af3f3-serving-cert\"" Nov 03 21:57:34.964540 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:34.963889 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-dg5g9\"" Nov 03 21:57:34.964540 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:34.964084 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Nov 03 21:57:34.964540 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:34.964271 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-8af3f3-8bcd46f45-qbvvd"] Nov 03 21:57:35.011735 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:35.011697 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1818211e-c9ac-4329-ad91-84825e22a424-maas-api-tls\") pod \"maas-api-e2e-worker-w6-8af3f3-8bcd46f45-qbvvd\" (UID: \"1818211e-c9ac-4329-ad91-84825e22a424\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-8af3f3-8bcd46f45-qbvvd" Nov 03 21:57:35.011735 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:35.011728 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1818211e-c9ac-4329-ad91-84825e22a424-metrics-certs\") pod \"maas-api-e2e-worker-w6-8af3f3-8bcd46f45-qbvvd\" (UID: \"1818211e-c9ac-4329-ad91-84825e22a424\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-8af3f3-8bcd46f45-qbvvd" Nov 03 21:57:35.011995 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:35.011763 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drb4t\" (UniqueName: \"kubernetes.io/projected/1818211e-c9ac-4329-ad91-84825e22a424-kube-api-access-drb4t\") pod \"maas-api-e2e-worker-w6-8af3f3-8bcd46f45-qbvvd\" (UID: \"1818211e-c9ac-4329-ad91-84825e22a424\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-8af3f3-8bcd46f45-qbvvd" Nov 03 21:57:35.012059 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:35.011981 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1818211e-c9ac-4329-ad91-84825e22a424-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-8af3f3-8bcd46f45-qbvvd\" (UID: \"1818211e-c9ac-4329-ad91-84825e22a424\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-8af3f3-8bcd46f45-qbvvd" Nov 03 21:57:35.113174 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:35.113111 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1818211e-c9ac-4329-ad91-84825e22a424-maas-api-tls\") pod \"maas-api-e2e-worker-w6-8af3f3-8bcd46f45-qbvvd\" (UID: \"1818211e-c9ac-4329-ad91-84825e22a424\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-8af3f3-8bcd46f45-qbvvd" Nov 03 21:57:35.113356 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:35.113182 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1818211e-c9ac-4329-ad91-84825e22a424-metrics-certs\") pod \"maas-api-e2e-worker-w6-8af3f3-8bcd46f45-qbvvd\" (UID: \"1818211e-c9ac-4329-ad91-84825e22a424\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-8af3f3-8bcd46f45-qbvvd" Nov 03 21:57:35.113356 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:35.113225 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drb4t\" (UniqueName: \"kubernetes.io/projected/1818211e-c9ac-4329-ad91-84825e22a424-kube-api-access-drb4t\") pod \"maas-api-e2e-worker-w6-8af3f3-8bcd46f45-qbvvd\" (UID: \"1818211e-c9ac-4329-ad91-84825e22a424\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-8af3f3-8bcd46f45-qbvvd" Nov 03 21:57:35.113356 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:35.113253 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1818211e-c9ac-4329-ad91-84825e22a424-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-8af3f3-8bcd46f45-qbvvd\" (UID: \"1818211e-c9ac-4329-ad91-84825e22a424\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-8af3f3-8bcd46f45-qbvvd" Nov 03 21:57:35.113926 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:35.113893 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1818211e-c9ac-4329-ad91-84825e22a424-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-8af3f3-8bcd46f45-qbvvd\" (UID: \"1818211e-c9ac-4329-ad91-84825e22a424\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-8af3f3-8bcd46f45-qbvvd" Nov 03 21:57:35.115549 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:35.115517 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1818211e-c9ac-4329-ad91-84825e22a424-maas-api-tls\") pod \"maas-api-e2e-worker-w6-8af3f3-8bcd46f45-qbvvd\" (UID: \"1818211e-c9ac-4329-ad91-84825e22a424\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-8af3f3-8bcd46f45-qbvvd" Nov 03 21:57:35.115671 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:35.115611 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1818211e-c9ac-4329-ad91-84825e22a424-metrics-certs\") pod \"maas-api-e2e-worker-w6-8af3f3-8bcd46f45-qbvvd\" (UID: \"1818211e-c9ac-4329-ad91-84825e22a424\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-8af3f3-8bcd46f45-qbvvd" Nov 03 21:57:35.125871 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:35.125840 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drb4t\" (UniqueName: \"kubernetes.io/projected/1818211e-c9ac-4329-ad91-84825e22a424-kube-api-access-drb4t\") pod \"maas-api-e2e-worker-w6-8af3f3-8bcd46f45-qbvvd\" (UID: \"1818211e-c9ac-4329-ad91-84825e22a424\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-8af3f3-8bcd46f45-qbvvd" Nov 03 21:57:35.271103 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:35.270990 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-8af3f3-8bcd46f45-qbvvd" Nov 03 21:57:35.574447 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:35.574360 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-8af3f3-8bcd46f45-qbvvd"] Nov 03 21:57:35.577230 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:57:35.577205 6524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1818211e_c9ac_4329_ad91_84825e22a424.slice/crio-2a30ce57d00c63020a7d224d4f6a5157b0e955038fcfc08b2836b796b750c4cb WatchSource:0}: Error finding container 2a30ce57d00c63020a7d224d4f6a5157b0e955038fcfc08b2836b796b750c4cb: Status 404 returned error can't find the container with id 2a30ce57d00c63020a7d224d4f6a5157b0e955038fcfc08b2836b796b750c4cb Nov 03 21:57:36.502786 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:36.502698 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-8af3f3-8bcd46f45-qbvvd" event={"ID":"1818211e-c9ac-4329-ad91-84825e22a424","Type":"ContainerStarted","Data":"2a30ce57d00c63020a7d224d4f6a5157b0e955038fcfc08b2836b796b750c4cb"} Nov 03 21:57:38.513304 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:38.513263 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-8af3f3-8bcd46f45-qbvvd" event={"ID":"1818211e-c9ac-4329-ad91-84825e22a424","Type":"ContainerStarted","Data":"5fa1571281442d07a73e4be32ebdc4ca451e9ca07f50d816aa8f33939d2f5d5f"} Nov 03 21:57:38.513785 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:38.513460 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-8af3f3-8bcd46f45-qbvvd" Nov 03 21:57:38.534186 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:38.534092 6524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-8af3f3-8bcd46f45-qbvvd" podStartSLOduration=2.060772355 podStartE2EDuration="4.534074939s" totalImagesPullingTime="2.473302584s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:57:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:57:38.533439182 +0000 UTC m=+703.885765996" watchObservedRunningTime="2026-11-03 21:57:38.534074939 +0000 UTC m=+703.886401747" Nov 03 21:57:44.524701 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:44.524668 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-8af3f3-8bcd46f45-qbvvd" Nov 03 21:57:45.516012 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:45.515976 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w6-8af3f3-655f44685b-wh9n2" Nov 03 21:57:45.516455 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:45.516430 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-8af3f3-7c9fcbb455-25425" Nov 03 21:57:49.568776 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:49.568684 6524 generic.go:356] "Generic (PLEG): container finished" podID="82c4e70e-4765-48d3-80d3-8e0311abfda7" containerID="c8e6e032a42781b491f35490e3e79e129f0f8aa6b69418c47760b39ef261dec2" exitCode=0 Nov 03 21:57:49.569334 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:49.568764 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" event={"ID":"82c4e70e-4765-48d3-80d3-8e0311abfda7","Type":"ContainerDied","Data":"c8e6e032a42781b491f35490e3e79e129f0f8aa6b69418c47760b39ef261dec2"} Nov 03 21:57:49.569334 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:49.569267 6524 scope.go:122] "RemoveContainer" containerID="c8e6e032a42781b491f35490e3e79e129f0f8aa6b69418c47760b39ef261dec2" Nov 03 21:57:49.570645 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:49.570617 6524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"} Nov 03 21:57:49.570742 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:49.570684 6524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"} Nov 03 21:57:49.570742 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:49.570715 6524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"} Nov 03 21:57:50.575635 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:50.575600 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" event={"ID":"82c4e70e-4765-48d3-80d3-8e0311abfda7","Type":"ContainerStarted","Data":"a3cc8b56794c604d5e10892d7544628cc525e88f823336a6f11c3011e76cb3cc"} Nov 03 21:57:51.222755 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:51.222711 6524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" Nov 03 21:57:51.223805 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:51.223774 6524 patch_prober.go:29] interesting pod/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body= Nov 03 21:57:51.223974 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:51.223858 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" podUID="82c4e70e-4765-48d3-80d3-8e0311abfda7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" Nov 03 21:57:52.222163 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:52.222090 6524 patch_prober.go:29] interesting pod/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body= Nov 03 21:57:52.222551 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:52.222178 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" podUID="82c4e70e-4765-48d3-80d3-8e0311abfda7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" Nov 03 21:57:53.222897 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:53.222860 6524 patch_prober.go:29] interesting pod/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body= Nov 03 21:57:53.223318 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:53.222919 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" podUID="82c4e70e-4765-48d3-80d3-8e0311abfda7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" Nov 03 21:57:54.222323 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:54.222290 6524 patch_prober.go:29] interesting pod/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body= Nov 03 21:57:54.222493 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:54.222341 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" podUID="82c4e70e-4765-48d3-80d3-8e0311abfda7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" Nov 03 21:57:54.434624 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:54.434575 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" Nov 03 21:57:55.222894 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:55.222853 6524 patch_prober.go:29] interesting pod/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body= Nov 03 21:57:55.223086 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:55.222908 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" podUID="82c4e70e-4765-48d3-80d3-8e0311abfda7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" Nov 03 21:57:56.226448 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:56.226413 6524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" Nov 03 21:57:56.227444 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:56.227417 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" Nov 03 21:57:56.369591 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:56.369555 6524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv"] Nov 03 21:57:56.375648 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:56.375621 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv" Nov 03 21:57:56.384537 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:56.384509 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv"] Nov 03 21:57:56.386470 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:56.386247 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-fc16de\"/\"openshift-service-ca.crt\"" Nov 03 21:57:56.386470 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:56.386343 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-fc16de\"/\"kube-root-ca.crt\"" Nov 03 21:57:56.386470 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:56.386450 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-fc16de\"/\"default-dockercfg-p6w68\"" Nov 03 21:57:56.386470 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:56.386455 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-fc16de\"/\"simulator-w4-fc16de-kserve-self-signed-certs\"" Nov 03 21:57:56.471455 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:56.471419 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/851596d1-02e4-42b9-bdab-19a9321a2a04-tls-certs\") pod \"simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv\" (UID: \"851596d1-02e4-42b9-bdab-19a9321a2a04\") " pod="e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv" Nov 03 21:57:56.471647 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:56.471471 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/851596d1-02e4-42b9-bdab-19a9321a2a04-tmp-dir\") pod \"simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv\" (UID: \"851596d1-02e4-42b9-bdab-19a9321a2a04\") " pod="e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv" Nov 03 21:57:56.471647 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:56.471543 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/851596d1-02e4-42b9-bdab-19a9321a2a04-model-cache\") pod \"simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv\" (UID: \"851596d1-02e4-42b9-bdab-19a9321a2a04\") " pod="e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv" Nov 03 21:57:56.471744 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:56.471681 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/851596d1-02e4-42b9-bdab-19a9321a2a04-dshm\") pod \"simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv\" (UID: \"851596d1-02e4-42b9-bdab-19a9321a2a04\") " pod="e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv" Nov 03 21:57:56.471804 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:56.471772 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/851596d1-02e4-42b9-bdab-19a9321a2a04-home\") pod \"simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv\" (UID: \"851596d1-02e4-42b9-bdab-19a9321a2a04\") " pod="e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv" Nov 03 21:57:56.471866 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:56.471814 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96fjc\" (UniqueName: \"kubernetes.io/projected/851596d1-02e4-42b9-bdab-19a9321a2a04-kube-api-access-96fjc\") pod \"simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv\" (UID: \"851596d1-02e4-42b9-bdab-19a9321a2a04\") " pod="e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv" Nov 03 21:57:56.572782 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:56.572691 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/851596d1-02e4-42b9-bdab-19a9321a2a04-dshm\") pod \"simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv\" (UID: \"851596d1-02e4-42b9-bdab-19a9321a2a04\") " pod="e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv" Nov 03 21:57:56.573611 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:56.573585 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/851596d1-02e4-42b9-bdab-19a9321a2a04-home\") pod \"simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv\" (UID: \"851596d1-02e4-42b9-bdab-19a9321a2a04\") " pod="e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv" Nov 03 21:57:56.573806 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:56.573787 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96fjc\" (UniqueName: \"kubernetes.io/projected/851596d1-02e4-42b9-bdab-19a9321a2a04-kube-api-access-96fjc\") pod \"simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv\" (UID: \"851596d1-02e4-42b9-bdab-19a9321a2a04\") " pod="e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv" Nov 03 21:57:56.573944 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:56.573931 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/851596d1-02e4-42b9-bdab-19a9321a2a04-tls-certs\") pod \"simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv\" (UID: \"851596d1-02e4-42b9-bdab-19a9321a2a04\") " pod="e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv" Nov 03 21:57:56.574108 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:56.574071 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/851596d1-02e4-42b9-bdab-19a9321a2a04-tmp-dir\") pod \"simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv\" (UID: \"851596d1-02e4-42b9-bdab-19a9321a2a04\") " pod="e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv" Nov 03 21:57:56.574302 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:56.574283 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/851596d1-02e4-42b9-bdab-19a9321a2a04-model-cache\") pod \"simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv\" (UID: \"851596d1-02e4-42b9-bdab-19a9321a2a04\") " pod="e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv" Nov 03 21:57:56.574872 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:56.574846 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/851596d1-02e4-42b9-bdab-19a9321a2a04-model-cache\") pod \"simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv\" (UID: \"851596d1-02e4-42b9-bdab-19a9321a2a04\") " pod="e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv" Nov 03 21:57:56.575187 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:56.575162 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/851596d1-02e4-42b9-bdab-19a9321a2a04-home\") pod \"simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv\" (UID: \"851596d1-02e4-42b9-bdab-19a9321a2a04\") " pod="e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv" Nov 03 21:57:56.575594 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:56.575551 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/851596d1-02e4-42b9-bdab-19a9321a2a04-tmp-dir\") pod \"simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv\" (UID: \"851596d1-02e4-42b9-bdab-19a9321a2a04\") " pod="e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv" Nov 03 21:57:56.578328 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:56.578300 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/851596d1-02e4-42b9-bdab-19a9321a2a04-dshm\") pod \"simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv\" (UID: \"851596d1-02e4-42b9-bdab-19a9321a2a04\") " pod="e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv" Nov 03 21:57:56.580266 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:56.580241 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/851596d1-02e4-42b9-bdab-19a9321a2a04-tls-certs\") pod \"simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv\" (UID: \"851596d1-02e4-42b9-bdab-19a9321a2a04\") " pod="e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv" Nov 03 21:57:56.593704 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:56.593269 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96fjc\" (UniqueName: \"kubernetes.io/projected/851596d1-02e4-42b9-bdab-19a9321a2a04-kube-api-access-96fjc\") pod \"simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv\" (UID: \"851596d1-02e4-42b9-bdab-19a9321a2a04\") " pod="e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv" Nov 03 21:57:56.691295 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:56.691261 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv" Nov 03 21:57:56.970745 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:56.970709 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv"] Nov 03 21:57:56.973819 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:57:56.973790 6524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod851596d1_02e4_42b9_bdab_19a9321a2a04.slice/crio-8f180beb1f86ab2301fc421a361b40920344a47f053dfe5e138bfb823649cb1d WatchSource:0}: Error finding container 8f180beb1f86ab2301fc421a361b40920344a47f053dfe5e138bfb823649cb1d: Status 404 returned error can't find the container with id 8f180beb1f86ab2301fc421a361b40920344a47f053dfe5e138bfb823649cb1d Nov 03 21:57:57.627218 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:57.627163 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv" event={"ID":"851596d1-02e4-42b9-bdab-19a9321a2a04","Type":"ContainerStarted","Data":"8f180beb1f86ab2301fc421a361b40920344a47f053dfe5e138bfb823649cb1d"} Nov 03 21:57:58.634252 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:58.634209 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv" event={"ID":"851596d1-02e4-42b9-bdab-19a9321a2a04","Type":"ContainerStarted","Data":"c582bc14a1410dc15a1fe39319b6e029db8853db184a89bb736565e45f67eaf6"} Nov 03 21:57:58.655157 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:58.655088 6524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv" podStartSLOduration=1.199817188 podStartE2EDuration="2.655075672s" totalImagesPullingTime="1.455258484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:57:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:57:58.654836978 +0000 UTC m=+724.007163783" watchObservedRunningTime="2026-11-03 21:57:58.655075672 +0000 UTC m=+724.007402494" Nov 03 21:57:58.691668 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:58.691468 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv" Nov 03 21:57:59.708927 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:57:59.708889 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 21:57:59.851592 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:59.851555 6524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf"] Nov 03 21:57:59.853968 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:59.853950 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf" Nov 03 21:57:59.859083 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:59.859058 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-c0caef\"/\"kube-root-ca.crt\"" Nov 03 21:57:59.860458 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:59.860418 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-c0caef\"/\"default-dockercfg-zzvmd\"" Nov 03 21:57:59.860581 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:59.860491 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-c0caef\"/\"openshift-service-ca.crt\"" Nov 03 21:57:59.860581 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:59.860496 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-c0caef\"/\"simulator-w0-c0caef-kserve-self-signed-certs\"" Nov 03 21:57:59.868784 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:59.868755 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf"] Nov 03 21:57:59.916333 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:59.916283 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ab01063-2737-4b29-ade3-ef9cc00ab5e1-model-cache\") pod \"simulator-w0-c0caef-kserve-5967669799-xtbxf\" (UID: \"1ab01063-2737-4b29-ade3-ef9cc00ab5e1\") " pod="e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf" Nov 03 21:57:59.916530 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:59.916343 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ab01063-2737-4b29-ade3-ef9cc00ab5e1-tmp-dir\") pod \"simulator-w0-c0caef-kserve-5967669799-xtbxf\" (UID: \"1ab01063-2737-4b29-ade3-ef9cc00ab5e1\") " pod="e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf" Nov 03 21:57:59.916530 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:59.916388 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ab01063-2737-4b29-ade3-ef9cc00ab5e1-tls-certs\") pod \"simulator-w0-c0caef-kserve-5967669799-xtbxf\" (UID: \"1ab01063-2737-4b29-ade3-ef9cc00ab5e1\") " pod="e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf" Nov 03 21:57:59.916530 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:59.916485 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ab01063-2737-4b29-ade3-ef9cc00ab5e1-dshm\") pod \"simulator-w0-c0caef-kserve-5967669799-xtbxf\" (UID: \"1ab01063-2737-4b29-ade3-ef9cc00ab5e1\") " pod="e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf" Nov 03 21:57:59.916530 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:59.916508 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cj2kh\" (UniqueName: \"kubernetes.io/projected/1ab01063-2737-4b29-ade3-ef9cc00ab5e1-kube-api-access-cj2kh\") pod \"simulator-w0-c0caef-kserve-5967669799-xtbxf\" (UID: \"1ab01063-2737-4b29-ade3-ef9cc00ab5e1\") " pod="e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf" Nov 03 21:57:59.916699 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:57:59.916565 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ab01063-2737-4b29-ade3-ef9cc00ab5e1-home\") pod \"simulator-w0-c0caef-kserve-5967669799-xtbxf\" (UID: \"1ab01063-2737-4b29-ade3-ef9cc00ab5e1\") " pod="e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf" Nov 03 21:58:00.017416 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:00.017328 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ab01063-2737-4b29-ade3-ef9cc00ab5e1-model-cache\") pod \"simulator-w0-c0caef-kserve-5967669799-xtbxf\" (UID: \"1ab01063-2737-4b29-ade3-ef9cc00ab5e1\") " pod="e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf" Nov 03 21:58:00.017416 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:00.017384 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ab01063-2737-4b29-ade3-ef9cc00ab5e1-tmp-dir\") pod \"simulator-w0-c0caef-kserve-5967669799-xtbxf\" (UID: \"1ab01063-2737-4b29-ade3-ef9cc00ab5e1\") " pod="e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf" Nov 03 21:58:00.017416 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:00.017412 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ab01063-2737-4b29-ade3-ef9cc00ab5e1-tls-certs\") pod \"simulator-w0-c0caef-kserve-5967669799-xtbxf\" (UID: \"1ab01063-2737-4b29-ade3-ef9cc00ab5e1\") " pod="e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf" Nov 03 21:58:00.017703 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:00.017444 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ab01063-2737-4b29-ade3-ef9cc00ab5e1-dshm\") pod \"simulator-w0-c0caef-kserve-5967669799-xtbxf\" (UID: \"1ab01063-2737-4b29-ade3-ef9cc00ab5e1\") " pod="e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf" Nov 03 21:58:00.017703 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:00.017478 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cj2kh\" (UniqueName: \"kubernetes.io/projected/1ab01063-2737-4b29-ade3-ef9cc00ab5e1-kube-api-access-cj2kh\") pod \"simulator-w0-c0caef-kserve-5967669799-xtbxf\" (UID: \"1ab01063-2737-4b29-ade3-ef9cc00ab5e1\") " pod="e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf" Nov 03 21:58:00.017703 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:00.017545 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ab01063-2737-4b29-ade3-ef9cc00ab5e1-home\") pod \"simulator-w0-c0caef-kserve-5967669799-xtbxf\" (UID: \"1ab01063-2737-4b29-ade3-ef9cc00ab5e1\") " pod="e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf" Nov 03 21:58:00.017864 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:00.017774 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ab01063-2737-4b29-ade3-ef9cc00ab5e1-model-cache\") pod \"simulator-w0-c0caef-kserve-5967669799-xtbxf\" (UID: \"1ab01063-2737-4b29-ade3-ef9cc00ab5e1\") " pod="e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf" Nov 03 21:58:00.017953 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:00.017927 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ab01063-2737-4b29-ade3-ef9cc00ab5e1-home\") pod \"simulator-w0-c0caef-kserve-5967669799-xtbxf\" (UID: \"1ab01063-2737-4b29-ade3-ef9cc00ab5e1\") " pod="e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf" Nov 03 21:58:00.018492 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:00.018467 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ab01063-2737-4b29-ade3-ef9cc00ab5e1-tmp-dir\") pod \"simulator-w0-c0caef-kserve-5967669799-xtbxf\" (UID: \"1ab01063-2737-4b29-ade3-ef9cc00ab5e1\") " pod="e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf" Nov 03 21:58:00.020464 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:00.020439 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ab01063-2737-4b29-ade3-ef9cc00ab5e1-dshm\") pod \"simulator-w0-c0caef-kserve-5967669799-xtbxf\" (UID: \"1ab01063-2737-4b29-ade3-ef9cc00ab5e1\") " pod="e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf" Nov 03 21:58:00.020559 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:00.020507 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ab01063-2737-4b29-ade3-ef9cc00ab5e1-tls-certs\") pod \"simulator-w0-c0caef-kserve-5967669799-xtbxf\" (UID: \"1ab01063-2737-4b29-ade3-ef9cc00ab5e1\") " pod="e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf" Nov 03 21:58:00.039165 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:00.038181 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cj2kh\" (UniqueName: \"kubernetes.io/projected/1ab01063-2737-4b29-ade3-ef9cc00ab5e1-kube-api-access-cj2kh\") pod \"simulator-w0-c0caef-kserve-5967669799-xtbxf\" (UID: \"1ab01063-2737-4b29-ade3-ef9cc00ab5e1\") " pod="e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf" Nov 03 21:58:00.167001 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:00.166965 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf" Nov 03 21:58:00.447720 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:00.447537 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf"] Nov 03 21:58:00.450344 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:58:00.450318 6524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1ab01063_2737_4b29_ade3_ef9cc00ab5e1.slice/crio-8b0fcd55ef182e0b222c24cfcf870215b9f5d910de993bfe370700c7fb15c7dc WatchSource:0}: Error finding container 8b0fcd55ef182e0b222c24cfcf870215b9f5d910de993bfe370700c7fb15c7dc: Status 404 returned error can't find the container with id 8b0fcd55ef182e0b222c24cfcf870215b9f5d910de993bfe370700c7fb15c7dc Nov 03 21:58:00.644553 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:00.644515 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf" event={"ID":"1ab01063-2737-4b29-ade3-ef9cc00ab5e1","Type":"ContainerStarted","Data":"fe389a2e12e1ea5918c87e8a762d1c1f8f166b701ce9d3b273797bd9e6d30358"} Nov 03 21:58:00.644751 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:00.644565 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf" event={"ID":"1ab01063-2737-4b29-ade3-ef9cc00ab5e1","Type":"ContainerStarted","Data":"8b0fcd55ef182e0b222c24cfcf870215b9f5d910de993bfe370700c7fb15c7dc"} Nov 03 21:58:00.664271 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:00.664171 6524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf" podStartSLOduration=1.664152558 podStartE2EDuration="1.664152558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:57:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:58:00.663789818 +0000 UTC m=+726.016116627" watchObservedRunningTime="2026-11-03 21:58:00.664152558 +0000 UTC m=+726.016479360" Nov 03 21:58:01.167514 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:01.167472 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf" Nov 03 21:58:06.692656 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:06.692431 6524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv" Nov 03 21:58:06.711559 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:06.711527 6524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv" Nov 03 21:58:07.703747 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:07.703719 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv" Nov 03 21:58:10.167723 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:10.167682 6524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf" Nov 03 21:58:10.180499 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:10.180461 6524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf" Nov 03 21:58:10.729112 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:10.729073 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf" Nov 03 21:58:20.652790 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:20.652753 6524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z"] Nov 03 21:58:20.656866 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:20.656836 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z" Nov 03 21:58:20.661500 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:20.661474 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-02e8ab\"/\"kube-root-ca.crt\"" Nov 03 21:58:20.662296 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:20.662257 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-02e8ab\"/\"openshift-service-ca.crt\"" Nov 03 21:58:20.662690 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:20.662663 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-02e8ab\"/\"premium-w3-02e8ab-kserve-self-signed-certs\"" Nov 03 21:58:20.663187 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:20.663166 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-02e8ab\"/\"default-dockercfg-77r2d\"" Nov 03 21:58:20.670998 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:20.670969 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z"] Nov 03 21:58:20.730472 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:20.730438 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0ef1acd-e793-4ea1-b89c-924d84be20bc-dshm\") pod \"premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z\" (UID: \"e0ef1acd-e793-4ea1-b89c-924d84be20bc\") " pod="e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z" Nov 03 21:58:20.730472 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:20.730474 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0ef1acd-e793-4ea1-b89c-924d84be20bc-model-cache\") pod \"premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z\" (UID: \"e0ef1acd-e793-4ea1-b89c-924d84be20bc\") " pod="e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z" Nov 03 21:58:20.730692 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:20.730511 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ww8t9\" (UniqueName: \"kubernetes.io/projected/e0ef1acd-e793-4ea1-b89c-924d84be20bc-kube-api-access-ww8t9\") pod \"premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z\" (UID: \"e0ef1acd-e793-4ea1-b89c-924d84be20bc\") " pod="e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z" Nov 03 21:58:20.730692 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:20.730658 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0ef1acd-e793-4ea1-b89c-924d84be20bc-tmp-dir\") pod \"premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z\" (UID: \"e0ef1acd-e793-4ea1-b89c-924d84be20bc\") " pod="e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z" Nov 03 21:58:20.730759 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:20.730694 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0ef1acd-e793-4ea1-b89c-924d84be20bc-tls-certs\") pod \"premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z\" (UID: \"e0ef1acd-e793-4ea1-b89c-924d84be20bc\") " pod="e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z" Nov 03 21:58:20.730851 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:20.730833 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0ef1acd-e793-4ea1-b89c-924d84be20bc-home\") pod \"premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z\" (UID: \"e0ef1acd-e793-4ea1-b89c-924d84be20bc\") " pod="e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z" Nov 03 21:58:20.837831 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:20.837778 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ww8t9\" (UniqueName: \"kubernetes.io/projected/e0ef1acd-e793-4ea1-b89c-924d84be20bc-kube-api-access-ww8t9\") pod \"premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z\" (UID: \"e0ef1acd-e793-4ea1-b89c-924d84be20bc\") " pod="e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z" Nov 03 21:58:20.838183 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:20.838146 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0ef1acd-e793-4ea1-b89c-924d84be20bc-tmp-dir\") pod \"premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z\" (UID: \"e0ef1acd-e793-4ea1-b89c-924d84be20bc\") " pod="e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z" Nov 03 21:58:20.838352 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:20.838335 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0ef1acd-e793-4ea1-b89c-924d84be20bc-tls-certs\") pod \"premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z\" (UID: \"e0ef1acd-e793-4ea1-b89c-924d84be20bc\") " pod="e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z" Nov 03 21:58:20.838509 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:20.838494 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0ef1acd-e793-4ea1-b89c-924d84be20bc-home\") pod \"premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z\" (UID: \"e0ef1acd-e793-4ea1-b89c-924d84be20bc\") " pod="e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z" Nov 03 21:58:20.838722 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:20.838706 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0ef1acd-e793-4ea1-b89c-924d84be20bc-dshm\") pod \"premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z\" (UID: \"e0ef1acd-e793-4ea1-b89c-924d84be20bc\") " pod="e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z" Nov 03 21:58:20.838886 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:20.838869 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0ef1acd-e793-4ea1-b89c-924d84be20bc-model-cache\") pod \"premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z\" (UID: \"e0ef1acd-e793-4ea1-b89c-924d84be20bc\") " pod="e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z" Nov 03 21:58:20.839572 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:20.839525 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0ef1acd-e793-4ea1-b89c-924d84be20bc-model-cache\") pod \"premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z\" (UID: \"e0ef1acd-e793-4ea1-b89c-924d84be20bc\") " pod="e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z" Nov 03 21:58:20.839791 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:20.839773 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0ef1acd-e793-4ea1-b89c-924d84be20bc-tmp-dir\") pod \"premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z\" (UID: \"e0ef1acd-e793-4ea1-b89c-924d84be20bc\") " pod="e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z" Nov 03 21:58:20.840600 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:20.840579 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0ef1acd-e793-4ea1-b89c-924d84be20bc-home\") pod \"premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z\" (UID: \"e0ef1acd-e793-4ea1-b89c-924d84be20bc\") " pod="e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z" Nov 03 21:58:20.842997 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:20.842969 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0ef1acd-e793-4ea1-b89c-924d84be20bc-tls-certs\") pod \"premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z\" (UID: \"e0ef1acd-e793-4ea1-b89c-924d84be20bc\") " pod="e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z" Nov 03 21:58:20.843314 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:20.843293 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0ef1acd-e793-4ea1-b89c-924d84be20bc-dshm\") pod \"premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z\" (UID: \"e0ef1acd-e793-4ea1-b89c-924d84be20bc\") " pod="e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z" Nov 03 21:58:20.856552 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:20.855921 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ww8t9\" (UniqueName: \"kubernetes.io/projected/e0ef1acd-e793-4ea1-b89c-924d84be20bc-kube-api-access-ww8t9\") pod \"premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z\" (UID: \"e0ef1acd-e793-4ea1-b89c-924d84be20bc\") " pod="e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z" Nov 03 21:58:20.977608 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:20.977514 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z" Nov 03 21:58:21.255082 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:21.254894 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z"] Nov 03 21:58:21.257945 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:58:21.257915 6524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode0ef1acd_e793_4ea1_b89c_924d84be20bc.slice/crio-2f45ac3107785dc2947a9a488d05fd4ad97a36dc59c67570a3c01516f8c7192d WatchSource:0}: Error finding container 2f45ac3107785dc2947a9a488d05fd4ad97a36dc59c67570a3c01516f8c7192d: Status 404 returned error can't find the container with id 2f45ac3107785dc2947a9a488d05fd4ad97a36dc59c67570a3c01516f8c7192d Nov 03 21:58:21.777143 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:21.777093 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z" event={"ID":"e0ef1acd-e793-4ea1-b89c-924d84be20bc","Type":"ContainerStarted","Data":"f494e9cda746fcb2a738c0f76e26e6e224be24694d21650def57269eb2c68d7c"} Nov 03 21:58:21.777646 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:21.777165 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z" event={"ID":"e0ef1acd-e793-4ea1-b89c-924d84be20bc","Type":"ContainerStarted","Data":"2f45ac3107785dc2947a9a488d05fd4ad97a36dc59c67570a3c01516f8c7192d"} Nov 03 21:58:21.798405 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:21.798354 6524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z" podStartSLOduration=1.798339377 podStartE2EDuration="1.798339377s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:58:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:58:21.797072669 +0000 UTC m=+747.149399468" watchObservedRunningTime="2026-11-03 21:58:21.798339377 +0000 UTC m=+747.150666180" Nov 03 21:58:21.978356 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:21.978319 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z" Nov 03 21:58:26.162845 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:26.162593 6524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j"] Nov 03 21:58:26.167321 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:26.167297 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j" Nov 03 21:58:26.172770 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:26.172742 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-f4fb1b\"/\"kube-root-ca.crt\"" Nov 03 21:58:26.175711 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:26.175335 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j"] Nov 03 21:58:26.175846 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:26.175821 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-f4fb1b\"/\"premium-w1-f4fb1b-kserve-self-signed-certs\"" Nov 03 21:58:26.175923 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:26.175766 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-f4fb1b\"/\"default-dockercfg-g9dgj\"" Nov 03 21:58:26.272587 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:26.272556 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-f4fb1b\"/\"openshift-service-ca.crt\"" Nov 03 21:58:26.309099 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:26.303918 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m98tz\" (UniqueName: \"kubernetes.io/projected/7df4ae2c-a36f-4933-9e10-1f712fb47595-kube-api-access-m98tz\") pod \"premium-w1-f4fb1b-kserve-5895cf468b-xg88j\" (UID: \"7df4ae2c-a36f-4933-9e10-1f712fb47595\") " pod="e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j" Nov 03 21:58:26.309099 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:26.303996 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7df4ae2c-a36f-4933-9e10-1f712fb47595-dshm\") pod \"premium-w1-f4fb1b-kserve-5895cf468b-xg88j\" (UID: \"7df4ae2c-a36f-4933-9e10-1f712fb47595\") " pod="e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j" Nov 03 21:58:26.309099 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:26.304190 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7df4ae2c-a36f-4933-9e10-1f712fb47595-tmp-dir\") pod \"premium-w1-f4fb1b-kserve-5895cf468b-xg88j\" (UID: \"7df4ae2c-a36f-4933-9e10-1f712fb47595\") " pod="e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j" Nov 03 21:58:26.309099 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:26.304223 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7df4ae2c-a36f-4933-9e10-1f712fb47595-home\") pod \"premium-w1-f4fb1b-kserve-5895cf468b-xg88j\" (UID: \"7df4ae2c-a36f-4933-9e10-1f712fb47595\") " pod="e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j" Nov 03 21:58:26.309099 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:26.304273 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7df4ae2c-a36f-4933-9e10-1f712fb47595-model-cache\") pod \"premium-w1-f4fb1b-kserve-5895cf468b-xg88j\" (UID: \"7df4ae2c-a36f-4933-9e10-1f712fb47595\") " pod="e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j" Nov 03 21:58:26.309099 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:26.304308 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7df4ae2c-a36f-4933-9e10-1f712fb47595-tls-certs\") pod \"premium-w1-f4fb1b-kserve-5895cf468b-xg88j\" (UID: \"7df4ae2c-a36f-4933-9e10-1f712fb47595\") " pod="e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j" Nov 03 21:58:26.405038 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:26.404993 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7df4ae2c-a36f-4933-9e10-1f712fb47595-model-cache\") pod \"premium-w1-f4fb1b-kserve-5895cf468b-xg88j\" (UID: \"7df4ae2c-a36f-4933-9e10-1f712fb47595\") " pod="e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j" Nov 03 21:58:26.405038 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:26.405045 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7df4ae2c-a36f-4933-9e10-1f712fb47595-tls-certs\") pod \"premium-w1-f4fb1b-kserve-5895cf468b-xg88j\" (UID: \"7df4ae2c-a36f-4933-9e10-1f712fb47595\") " pod="e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j" Nov 03 21:58:26.405317 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:26.405080 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m98tz\" (UniqueName: \"kubernetes.io/projected/7df4ae2c-a36f-4933-9e10-1f712fb47595-kube-api-access-m98tz\") pod \"premium-w1-f4fb1b-kserve-5895cf468b-xg88j\" (UID: \"7df4ae2c-a36f-4933-9e10-1f712fb47595\") " pod="e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j" Nov 03 21:58:26.405317 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:26.405154 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7df4ae2c-a36f-4933-9e10-1f712fb47595-dshm\") pod \"premium-w1-f4fb1b-kserve-5895cf468b-xg88j\" (UID: \"7df4ae2c-a36f-4933-9e10-1f712fb47595\") " pod="e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j" Nov 03 21:58:26.405317 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:26.405247 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7df4ae2c-a36f-4933-9e10-1f712fb47595-tmp-dir\") pod \"premium-w1-f4fb1b-kserve-5895cf468b-xg88j\" (UID: \"7df4ae2c-a36f-4933-9e10-1f712fb47595\") " pod="e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j" Nov 03 21:58:26.405317 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:26.405265 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7df4ae2c-a36f-4933-9e10-1f712fb47595-home\") pod \"premium-w1-f4fb1b-kserve-5895cf468b-xg88j\" (UID: \"7df4ae2c-a36f-4933-9e10-1f712fb47595\") " pod="e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j" Nov 03 21:58:26.405561 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:26.405531 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7df4ae2c-a36f-4933-9e10-1f712fb47595-model-cache\") pod \"premium-w1-f4fb1b-kserve-5895cf468b-xg88j\" (UID: \"7df4ae2c-a36f-4933-9e10-1f712fb47595\") " pod="e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j" Nov 03 21:58:26.405694 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:26.405616 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7df4ae2c-a36f-4933-9e10-1f712fb47595-home\") pod \"premium-w1-f4fb1b-kserve-5895cf468b-xg88j\" (UID: \"7df4ae2c-a36f-4933-9e10-1f712fb47595\") " pod="e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j" Nov 03 21:58:26.405789 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:26.405768 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7df4ae2c-a36f-4933-9e10-1f712fb47595-tmp-dir\") pod \"premium-w1-f4fb1b-kserve-5895cf468b-xg88j\" (UID: \"7df4ae2c-a36f-4933-9e10-1f712fb47595\") " pod="e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j" Nov 03 21:58:26.407537 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:26.407516 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7df4ae2c-a36f-4933-9e10-1f712fb47595-dshm\") pod \"premium-w1-f4fb1b-kserve-5895cf468b-xg88j\" (UID: \"7df4ae2c-a36f-4933-9e10-1f712fb47595\") " pod="e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j" Nov 03 21:58:26.407771 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:26.407751 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7df4ae2c-a36f-4933-9e10-1f712fb47595-tls-certs\") pod \"premium-w1-f4fb1b-kserve-5895cf468b-xg88j\" (UID: \"7df4ae2c-a36f-4933-9e10-1f712fb47595\") " pod="e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j" Nov 03 21:58:26.417811 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:26.417750 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m98tz\" (UniqueName: \"kubernetes.io/projected/7df4ae2c-a36f-4933-9e10-1f712fb47595-kube-api-access-m98tz\") pod \"premium-w1-f4fb1b-kserve-5895cf468b-xg88j\" (UID: \"7df4ae2c-a36f-4933-9e10-1f712fb47595\") " pod="e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j" Nov 03 21:58:26.481573 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:26.481537 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j" Nov 03 21:58:26.758495 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:26.758464 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j"] Nov 03 21:58:26.761254 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:58:26.761220 6524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7df4ae2c_a36f_4933_9e10_1f712fb47595.slice/crio-a14989a22df6d2af6f482433a0fef70df1da24b1dc3302fa134d1bc17a6c77b5 WatchSource:0}: Error finding container a14989a22df6d2af6f482433a0fef70df1da24b1dc3302fa134d1bc17a6c77b5: Status 404 returned error can't find the container with id a14989a22df6d2af6f482433a0fef70df1da24b1dc3302fa134d1bc17a6c77b5 Nov 03 21:58:26.810314 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:26.810282 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j" event={"ID":"7df4ae2c-a36f-4933-9e10-1f712fb47595","Type":"ContainerStarted","Data":"a14989a22df6d2af6f482433a0fef70df1da24b1dc3302fa134d1bc17a6c77b5"} Nov 03 21:58:27.817088 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:27.817048 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j" event={"ID":"7df4ae2c-a36f-4933-9e10-1f712fb47595","Type":"ContainerStarted","Data":"132404f1eaaac3fcb29279a843e96df653ca4ffafddfab697c556919b5f4a519"} Nov 03 21:58:27.837990 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:27.837929 6524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j" podStartSLOduration=1.837911166 podStartE2EDuration="1.837911166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:58:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:58:27.836935514 +0000 UTC m=+753.189262340" watchObservedRunningTime="2026-11-03 21:58:27.837911166 +0000 UTC m=+753.190237967" Nov 03 21:58:28.482275 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:28.482232 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j" Nov 03 21:58:29.708254 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:58:29.708216 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 21:58:30.978683 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:30.978649 6524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z" Nov 03 21:58:30.991432 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:30.991406 6524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z" Nov 03 21:58:31.850020 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:31.849988 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z" Nov 03 21:58:36.482208 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:36.482169 6524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j" Nov 03 21:58:36.495425 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:36.495393 6524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j" Nov 03 21:58:36.871932 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:36.871900 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j" Nov 03 21:58:51.773320 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:51.773283 6524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9"] Nov 03 21:58:51.776756 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:51.776736 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" Nov 03 21:58:51.783300 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:51.783269 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-02e8ab-openshift-default-dockercfg-4kgqm\"" Nov 03 21:58:51.797051 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:51.797016 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9"] Nov 03 21:58:51.882158 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:51.880104 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8462643c-89e1-4ee3-a413-44160da8e03d-istio-envoy\") pod \"e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9\" (UID: \"8462643c-89e1-4ee3-a413-44160da8e03d\") " pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" Nov 03 21:58:51.882158 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:51.880215 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8462643c-89e1-4ee3-a413-44160da8e03d-workload-socket\") pod \"e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9\" (UID: \"8462643c-89e1-4ee3-a413-44160da8e03d\") " pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" Nov 03 21:58:51.882158 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:51.880251 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8462643c-89e1-4ee3-a413-44160da8e03d-istiod-ca-cert\") pod \"e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9\" (UID: \"8462643c-89e1-4ee3-a413-44160da8e03d\") " pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" Nov 03 21:58:51.882158 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:51.880305 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8462643c-89e1-4ee3-a413-44160da8e03d-credential-socket\") pod \"e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9\" (UID: \"8462643c-89e1-4ee3-a413-44160da8e03d\") " pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" Nov 03 21:58:51.882158 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:51.880340 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26lhf\" (UniqueName: \"kubernetes.io/projected/8462643c-89e1-4ee3-a413-44160da8e03d-kube-api-access-26lhf\") pod \"e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9\" (UID: \"8462643c-89e1-4ee3-a413-44160da8e03d\") " pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" Nov 03 21:58:51.882158 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:51.880412 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8462643c-89e1-4ee3-a413-44160da8e03d-istio-podinfo\") pod \"e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9\" (UID: \"8462643c-89e1-4ee3-a413-44160da8e03d\") " pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" Nov 03 21:58:51.882158 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:51.880445 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8462643c-89e1-4ee3-a413-44160da8e03d-istio-data\") pod \"e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9\" (UID: \"8462643c-89e1-4ee3-a413-44160da8e03d\") " pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" Nov 03 21:58:51.882158 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:51.880472 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8462643c-89e1-4ee3-a413-44160da8e03d-workload-certs\") pod \"e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9\" (UID: \"8462643c-89e1-4ee3-a413-44160da8e03d\") " pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" Nov 03 21:58:51.882158 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:51.880516 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8462643c-89e1-4ee3-a413-44160da8e03d-istio-token\") pod \"e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9\" (UID: \"8462643c-89e1-4ee3-a413-44160da8e03d\") " pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" Nov 03 21:58:51.883144 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:51.882912 6524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2"] Nov 03 21:58:51.898480 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:51.898446 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" Nov 03 21:58:51.901275 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:51.899551 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2"] Nov 03 21:58:51.908723 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:51.908570 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-85b4ff-openshift-default-dockercfg-m2k4x\"" Nov 03 21:58:51.981654 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:51.981622 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8462643c-89e1-4ee3-a413-44160da8e03d-credential-socket\") pod \"e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9\" (UID: \"8462643c-89e1-4ee3-a413-44160da8e03d\") " pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" Nov 03 21:58:51.981844 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:51.981660 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26lhf\" (UniqueName: \"kubernetes.io/projected/8462643c-89e1-4ee3-a413-44160da8e03d-kube-api-access-26lhf\") pod \"e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9\" (UID: \"8462643c-89e1-4ee3-a413-44160da8e03d\") " pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" Nov 03 21:58:51.981844 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:51.981685 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqr48\" (UniqueName: \"kubernetes.io/projected/7a83deb0-69e5-4c9d-a573-235be3ff17ef-kube-api-access-zqr48\") pod \"e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2\" (UID: \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\") " pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" Nov 03 21:58:51.981844 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:51.981810 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a83deb0-69e5-4c9d-a573-235be3ff17ef-istio-token\") pod \"e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2\" (UID: \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\") " pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" Nov 03 21:58:51.981979 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:51.981874 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a83deb0-69e5-4c9d-a573-235be3ff17ef-istiod-ca-cert\") pod \"e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2\" (UID: \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\") " pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" Nov 03 21:58:51.981979 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:51.981945 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8462643c-89e1-4ee3-a413-44160da8e03d-istio-podinfo\") pod \"e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9\" (UID: \"8462643c-89e1-4ee3-a413-44160da8e03d\") " pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" Nov 03 21:58:51.981979 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:51.981975 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8462643c-89e1-4ee3-a413-44160da8e03d-istio-data\") pod \"e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9\" (UID: \"8462643c-89e1-4ee3-a413-44160da8e03d\") " pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" Nov 03 21:58:51.982119 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:51.981991 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a83deb0-69e5-4c9d-a573-235be3ff17ef-credential-socket\") pod \"e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2\" (UID: \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\") " pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" Nov 03 21:58:51.982119 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:51.982013 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8462643c-89e1-4ee3-a413-44160da8e03d-workload-certs\") pod \"e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9\" (UID: \"8462643c-89e1-4ee3-a413-44160da8e03d\") " pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" Nov 03 21:58:51.982119 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:51.982063 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8462643c-89e1-4ee3-a413-44160da8e03d-credential-socket\") pod \"e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9\" (UID: \"8462643c-89e1-4ee3-a413-44160da8e03d\") " pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" Nov 03 21:58:51.982295 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:51.982159 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a83deb0-69e5-4c9d-a573-235be3ff17ef-istio-podinfo\") pod \"e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2\" (UID: \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\") " pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" Nov 03 21:58:51.982295 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:51.982248 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8462643c-89e1-4ee3-a413-44160da8e03d-istio-token\") pod \"e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9\" (UID: \"8462643c-89e1-4ee3-a413-44160da8e03d\") " pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" Nov 03 21:58:51.982401 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:51.982291 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8462643c-89e1-4ee3-a413-44160da8e03d-istio-data\") pod \"e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9\" (UID: \"8462643c-89e1-4ee3-a413-44160da8e03d\") " pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" Nov 03 21:58:51.982401 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:51.982322 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8462643c-89e1-4ee3-a413-44160da8e03d-istio-envoy\") pod \"e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9\" (UID: \"8462643c-89e1-4ee3-a413-44160da8e03d\") " pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" Nov 03 21:58:51.982401 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:51.982365 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a83deb0-69e5-4c9d-a573-235be3ff17ef-workload-socket\") pod \"e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2\" (UID: \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\") " pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" Nov 03 21:58:51.982557 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:51.982400 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a83deb0-69e5-4c9d-a573-235be3ff17ef-workload-certs\") pod \"e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2\" (UID: \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\") " pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" Nov 03 21:58:51.982557 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:51.982415 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8462643c-89e1-4ee3-a413-44160da8e03d-workload-certs\") pod \"e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9\" (UID: \"8462643c-89e1-4ee3-a413-44160da8e03d\") " pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" Nov 03 21:58:51.982557 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:51.982433 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a83deb0-69e5-4c9d-a573-235be3ff17ef-istio-data\") pod \"e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2\" (UID: \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\") " pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" Nov 03 21:58:51.982557 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:51.982486 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a83deb0-69e5-4c9d-a573-235be3ff17ef-istio-envoy\") pod \"e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2\" (UID: \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\") " pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" Nov 03 21:58:51.982557 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:51.982548 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8462643c-89e1-4ee3-a413-44160da8e03d-workload-socket\") pod \"e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9\" (UID: \"8462643c-89e1-4ee3-a413-44160da8e03d\") " pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" Nov 03 21:58:51.982799 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:51.982569 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8462643c-89e1-4ee3-a413-44160da8e03d-istiod-ca-cert\") pod \"e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9\" (UID: \"8462643c-89e1-4ee3-a413-44160da8e03d\") " pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" Nov 03 21:58:51.982871 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:51.982848 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8462643c-89e1-4ee3-a413-44160da8e03d-workload-socket\") pod \"e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9\" (UID: \"8462643c-89e1-4ee3-a413-44160da8e03d\") " pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" Nov 03 21:58:51.983165 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:51.983146 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8462643c-89e1-4ee3-a413-44160da8e03d-istiod-ca-cert\") pod \"e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9\" (UID: \"8462643c-89e1-4ee3-a413-44160da8e03d\") " pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" Nov 03 21:58:51.984681 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:51.984655 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8462643c-89e1-4ee3-a413-44160da8e03d-istio-podinfo\") pod \"e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9\" (UID: \"8462643c-89e1-4ee3-a413-44160da8e03d\") " pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" Nov 03 21:58:51.984781 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:51.984665 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8462643c-89e1-4ee3-a413-44160da8e03d-istio-envoy\") pod \"e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9\" (UID: \"8462643c-89e1-4ee3-a413-44160da8e03d\") " pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" Nov 03 21:58:52.000810 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:52.000786 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-26lhf\" (UniqueName: \"kubernetes.io/projected/8462643c-89e1-4ee3-a413-44160da8e03d-kube-api-access-26lhf\") pod \"e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9\" (UID: \"8462643c-89e1-4ee3-a413-44160da8e03d\") " pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" Nov 03 21:58:52.000922 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:52.000837 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8462643c-89e1-4ee3-a413-44160da8e03d-istio-token\") pod \"e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9\" (UID: \"8462643c-89e1-4ee3-a413-44160da8e03d\") " pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" Nov 03 21:58:52.083435 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:52.083395 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a83deb0-69e5-4c9d-a573-235be3ff17ef-credential-socket\") pod \"e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2\" (UID: \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\") " pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" Nov 03 21:58:52.083628 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:52.083444 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a83deb0-69e5-4c9d-a573-235be3ff17ef-istio-podinfo\") pod \"e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2\" (UID: \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\") " pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" Nov 03 21:58:52.083628 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:52.083503 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a83deb0-69e5-4c9d-a573-235be3ff17ef-workload-socket\") pod \"e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2\" (UID: \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\") " pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" Nov 03 21:58:52.083628 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:52.083531 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a83deb0-69e5-4c9d-a573-235be3ff17ef-workload-certs\") pod \"e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2\" (UID: \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\") " pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" Nov 03 21:58:52.083628 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:52.083562 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a83deb0-69e5-4c9d-a573-235be3ff17ef-istio-data\") pod \"e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2\" (UID: \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\") " pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" Nov 03 21:58:52.083628 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:52.083585 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a83deb0-69e5-4c9d-a573-235be3ff17ef-istio-envoy\") pod \"e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2\" (UID: \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\") " pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" Nov 03 21:58:52.085028 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:52.083665 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqr48\" (UniqueName: \"kubernetes.io/projected/7a83deb0-69e5-4c9d-a573-235be3ff17ef-kube-api-access-zqr48\") pod \"e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2\" (UID: \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\") " pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" Nov 03 21:58:52.085028 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:52.083694 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a83deb0-69e5-4c9d-a573-235be3ff17ef-istio-token\") pod \"e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2\" (UID: \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\") " pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" Nov 03 21:58:52.085028 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:52.083949 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a83deb0-69e5-4c9d-a573-235be3ff17ef-istiod-ca-cert\") pod \"e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2\" (UID: \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\") " pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" Nov 03 21:58:52.085028 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:52.084196 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a83deb0-69e5-4c9d-a573-235be3ff17ef-credential-socket\") pod \"e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2\" (UID: \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\") " pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" Nov 03 21:58:52.085028 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:52.084218 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a83deb0-69e5-4c9d-a573-235be3ff17ef-istio-data\") pod \"e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2\" (UID: \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\") " pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" Nov 03 21:58:52.085028 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:52.084293 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a83deb0-69e5-4c9d-a573-235be3ff17ef-workload-certs\") pod \"e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2\" (UID: \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\") " pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" Nov 03 21:58:52.085028 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:52.084331 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a83deb0-69e5-4c9d-a573-235be3ff17ef-workload-socket\") pod \"e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2\" (UID: \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\") " pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" Nov 03 21:58:52.085028 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:52.084758 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a83deb0-69e5-4c9d-a573-235be3ff17ef-istiod-ca-cert\") pod \"e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2\" (UID: \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\") " pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" Nov 03 21:58:52.086023 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:52.085993 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a83deb0-69e5-4c9d-a573-235be3ff17ef-istio-envoy\") pod \"e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2\" (UID: \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\") " pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" Nov 03 21:58:52.086229 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:52.086205 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a83deb0-69e5-4c9d-a573-235be3ff17ef-istio-podinfo\") pod \"e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2\" (UID: \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\") " pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" Nov 03 21:58:52.091090 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:52.091069 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" Nov 03 21:58:52.098841 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:52.097607 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a83deb0-69e5-4c9d-a573-235be3ff17ef-istio-token\") pod \"e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2\" (UID: \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\") " pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" Nov 03 21:58:52.098841 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:52.097893 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqr48\" (UniqueName: \"kubernetes.io/projected/7a83deb0-69e5-4c9d-a573-235be3ff17ef-kube-api-access-zqr48\") pod \"e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2\" (UID: \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\") " pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" Nov 03 21:58:52.215271 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:52.215232 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" Nov 03 21:58:52.376450 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:52.376416 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9"] Nov 03 21:58:52.384846 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:58:52.384804 6524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8462643c_89e1_4ee3_a413_44160da8e03d.slice/crio-bdb7039e3dfd64fdfaa52141d5e4680d1b28b96bf9d0988322e42ee644881e5f WatchSource:0}: Error finding container bdb7039e3dfd64fdfaa52141d5e4680d1b28b96bf9d0988322e42ee644881e5f: Status 404 returned error can't find the container with id bdb7039e3dfd64fdfaa52141d5e4680d1b28b96bf9d0988322e42ee644881e5f Nov 03 21:58:52.387788 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:52.387508 6524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"} Nov 03 21:58:52.387788 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:52.387651 6524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"} Nov 03 21:58:52.387788 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:52.387731 6524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"} Nov 03 21:58:52.494030 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:52.493999 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2"] Nov 03 21:58:52.497156 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:58:52.497114 6524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a83deb0_69e5_4c9d_a573_235be3ff17ef.slice/crio-00982e4bee9dbb6e97b337ed0b094aeeba940fda5aeff1b1d35934aed4d5c0db WatchSource:0}: Error finding container 00982e4bee9dbb6e97b337ed0b094aeeba940fda5aeff1b1d35934aed4d5c0db: Status 404 returned error can't find the container with id 00982e4bee9dbb6e97b337ed0b094aeeba940fda5aeff1b1d35934aed4d5c0db Nov 03 21:58:52.499546 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:52.499512 6524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"} Nov 03 21:58:52.499659 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:52.499600 6524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"} Nov 03 21:58:52.499659 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:52.499645 6524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"} Nov 03 21:58:52.935296 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:52.935253 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" event={"ID":"7a83deb0-69e5-4c9d-a573-235be3ff17ef","Type":"ContainerStarted","Data":"640fe0bf5b0c90c23ecf2082960fe45c8714919d32574ca3ee7751064084e229"} Nov 03 21:58:52.935783 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:52.935317 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" event={"ID":"7a83deb0-69e5-4c9d-a573-235be3ff17ef","Type":"ContainerStarted","Data":"00982e4bee9dbb6e97b337ed0b094aeeba940fda5aeff1b1d35934aed4d5c0db"} Nov 03 21:58:52.937281 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:52.937251 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" event={"ID":"8462643c-89e1-4ee3-a413-44160da8e03d","Type":"ContainerStarted","Data":"3e1d409fe806d4bbaba0b8b39ab4c5c332583086ee3c8a99cb87a9fc7796744e"} Nov 03 21:58:52.937452 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:52.937312 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" event={"ID":"8462643c-89e1-4ee3-a413-44160da8e03d","Type":"ContainerStarted","Data":"bdb7039e3dfd64fdfaa52141d5e4680d1b28b96bf9d0988322e42ee644881e5f"} Nov 03 21:58:52.959740 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:52.959673 6524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" podStartSLOduration=1.9596549319999998 podStartE2EDuration="1.959654932s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:58:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:58:52.958599965 +0000 UTC m=+778.310926792" watchObservedRunningTime="2026-11-03 21:58:52.959654932 +0000 UTC m=+778.311981736" Nov 03 21:58:52.986033 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:52.985972 6524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" podStartSLOduration=1.985953834 podStartE2EDuration="1.985953834s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:58:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:58:52.985536939 +0000 UTC m=+778.337863745" watchObservedRunningTime="2026-11-03 21:58:52.985953834 +0000 UTC m=+778.338280640" Nov 03 21:58:53.092314 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:53.092271 6524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" Nov 03 21:58:53.093989 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:53.093960 6524 patch_prober.go:29] interesting pod/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body= Nov 03 21:58:53.094141 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:53.094021 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" podUID="8462643c-89e1-4ee3-a413-44160da8e03d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" Nov 03 21:58:53.222218 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:53.217490 6524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" Nov 03 21:58:53.222404 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:53.222355 6524 patch_prober.go:29] interesting pod/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Nov 03 21:58:53.222472 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:53.222433 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" podUID="7a83deb0-69e5-4c9d-a573-235be3ff17ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Nov 03 21:58:54.109148 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:54.098753 6524 patch_prober.go:29] interesting pod/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body= Nov 03 21:58:54.109148 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:54.098835 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" podUID="8462643c-89e1-4ee3-a413-44160da8e03d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" Nov 03 21:58:54.218157 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:54.217812 6524 patch_prober.go:29] interesting pod/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Nov 03 21:58:54.218157 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:54.217875 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" podUID="7a83deb0-69e5-4c9d-a573-235be3ff17ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Nov 03 21:58:55.107161 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:55.106548 6524 patch_prober.go:29] interesting pod/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body= Nov 03 21:58:55.107161 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:55.106611 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" podUID="8462643c-89e1-4ee3-a413-44160da8e03d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" Nov 03 21:58:55.234806 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:55.234308 6524 patch_prober.go:29] interesting pod/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Nov 03 21:58:55.234806 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:55.234376 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" podUID="7a83deb0-69e5-4c9d-a573-235be3ff17ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Nov 03 21:58:56.093591 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:56.093551 6524 patch_prober.go:29] interesting pod/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body= Nov 03 21:58:56.093899 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:56.093869 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" podUID="8462643c-89e1-4ee3-a413-44160da8e03d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" Nov 03 21:58:56.220084 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:56.220043 6524 patch_prober.go:29] interesting pod/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Nov 03 21:58:56.220401 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:56.220369 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" podUID="7a83deb0-69e5-4c9d-a573-235be3ff17ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Nov 03 21:58:56.694393 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:56.694352 6524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d5888686-q6tdc"] Nov 03 21:58:56.727419 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:56.727386 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d5888686-q6tdc"] Nov 03 21:58:56.727808 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:56.727791 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5888686-q6tdc" Nov 03 21:58:56.746237 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:56.746189 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Nov 03 21:58:56.747383 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:56.747321 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5n7j7\"" Nov 03 21:58:56.811147 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:56.810724 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1eabd95e-9b87-42d7-a442-e7cef3a2c73f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d5888686-q6tdc\" (UID: \"1eabd95e-9b87-42d7-a442-e7cef3a2c73f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5888686-q6tdc" Nov 03 21:58:56.811147 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:56.810860 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1eabd95e-9b87-42d7-a442-e7cef3a2c73f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d5888686-q6tdc\" (UID: \"1eabd95e-9b87-42d7-a442-e7cef3a2c73f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5888686-q6tdc" Nov 03 21:58:56.811147 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:56.810926 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stwn8\" (UniqueName: \"kubernetes.io/projected/1eabd95e-9b87-42d7-a442-e7cef3a2c73f-kube-api-access-stwn8\") pod \"maas-api-revoke-sub-d5888686-q6tdc\" (UID: \"1eabd95e-9b87-42d7-a442-e7cef3a2c73f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5888686-q6tdc" Nov 03 21:58:56.911753 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:56.911707 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1eabd95e-9b87-42d7-a442-e7cef3a2c73f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d5888686-q6tdc\" (UID: \"1eabd95e-9b87-42d7-a442-e7cef3a2c73f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5888686-q6tdc" Nov 03 21:58:56.912097 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:56.912071 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stwn8\" (UniqueName: \"kubernetes.io/projected/1eabd95e-9b87-42d7-a442-e7cef3a2c73f-kube-api-access-stwn8\") pod \"maas-api-revoke-sub-d5888686-q6tdc\" (UID: \"1eabd95e-9b87-42d7-a442-e7cef3a2c73f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5888686-q6tdc" Nov 03 21:58:56.912335 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:56.912315 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1eabd95e-9b87-42d7-a442-e7cef3a2c73f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d5888686-q6tdc\" (UID: \"1eabd95e-9b87-42d7-a442-e7cef3a2c73f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5888686-q6tdc" Nov 03 21:58:56.917156 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:56.914102 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1eabd95e-9b87-42d7-a442-e7cef3a2c73f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d5888686-q6tdc\" (UID: \"1eabd95e-9b87-42d7-a442-e7cef3a2c73f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5888686-q6tdc" Nov 03 21:58:56.936060 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:56.935979 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1eabd95e-9b87-42d7-a442-e7cef3a2c73f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d5888686-q6tdc\" (UID: \"1eabd95e-9b87-42d7-a442-e7cef3a2c73f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5888686-q6tdc" Nov 03 21:58:56.936948 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:56.936901 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-stwn8\" (UniqueName: \"kubernetes.io/projected/1eabd95e-9b87-42d7-a442-e7cef3a2c73f-kube-api-access-stwn8\") pod \"maas-api-revoke-sub-d5888686-q6tdc\" (UID: \"1eabd95e-9b87-42d7-a442-e7cef3a2c73f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5888686-q6tdc" Nov 03 21:58:57.062408 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:57.062315 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5888686-q6tdc" Nov 03 21:58:57.113150 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:57.110578 6524 patch_prober.go:29] interesting pod/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body= Nov 03 21:58:57.113150 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:57.110685 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" podUID="8462643c-89e1-4ee3-a413-44160da8e03d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" Nov 03 21:58:57.229149 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:57.215686 6524 patch_prober.go:29] interesting pod/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Nov 03 21:58:57.229149 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:57.215750 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" podUID="7a83deb0-69e5-4c9d-a573-235be3ff17ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Nov 03 21:58:57.391812 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:57.391773 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d5888686-q6tdc"] Nov 03 21:58:57.961772 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:57.961668 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5888686-q6tdc" event={"ID":"1eabd95e-9b87-42d7-a442-e7cef3a2c73f","Type":"ContainerStarted","Data":"3a81b2fcd83d5d8bec5db1ad68e86302e0d11f0da8f0155921b1ae48b5cb8d91"} Nov 03 21:58:58.094610 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:58.094138 6524 patch_prober.go:29] interesting pod/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body= Nov 03 21:58:58.094610 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:58.094207 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" podUID="8462643c-89e1-4ee3-a413-44160da8e03d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" Nov 03 21:58:58.221292 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:58.221172 6524 patch_prober.go:29] interesting pod/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Nov 03 21:58:58.221552 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:58.221508 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" podUID="7a83deb0-69e5-4c9d-a573-235be3ff17ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Nov 03 21:58:59.092099 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:59.091692 6524 patch_prober.go:29] interesting pod/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body= Nov 03 21:58:59.092099 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:59.091755 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" podUID="8462643c-89e1-4ee3-a413-44160da8e03d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" Nov 03 21:58:59.157580 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:59.157540 6524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-61f394fa-s7wth"] Nov 03 21:58:59.161713 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:59.161685 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-61f394fa-s7wth" Nov 03 21:58:59.181079 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:59.180800 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2d7sg\" (UniqueName: \"kubernetes.io/projected/5b3c5212-f9b9-4845-aab7-a8aac4eacc87-kube-api-access-2d7sg\") pod \"maas-api-revoke-sub-61f394fa-s7wth\" (UID: \"5b3c5212-f9b9-4845-aab7-a8aac4eacc87\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-61f394fa-s7wth" Nov 03 21:58:59.181079 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:59.180911 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5b3c5212-f9b9-4845-aab7-a8aac4eacc87-maas-api-xks-ca\") pod \"maas-api-revoke-sub-61f394fa-s7wth\" (UID: \"5b3c5212-f9b9-4845-aab7-a8aac4eacc87\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-61f394fa-s7wth" Nov 03 21:58:59.181079 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:59.180948 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5b3c5212-f9b9-4845-aab7-a8aac4eacc87-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-61f394fa-s7wth\" (UID: \"5b3c5212-f9b9-4845-aab7-a8aac4eacc87\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-61f394fa-s7wth" Nov 03 21:58:59.182545 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:59.182515 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-61f394fa-s7wth"] Nov 03 21:58:59.215999 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:59.215963 6524 patch_prober.go:29] interesting pod/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Nov 03 21:58:59.216208 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:59.216014 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" podUID="7a83deb0-69e5-4c9d-a573-235be3ff17ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Nov 03 21:58:59.282478 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:59.282440 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5b3c5212-f9b9-4845-aab7-a8aac4eacc87-maas-api-xks-ca\") pod \"maas-api-revoke-sub-61f394fa-s7wth\" (UID: \"5b3c5212-f9b9-4845-aab7-a8aac4eacc87\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-61f394fa-s7wth" Nov 03 21:58:59.282675 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:59.282493 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5b3c5212-f9b9-4845-aab7-a8aac4eacc87-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-61f394fa-s7wth\" (UID: \"5b3c5212-f9b9-4845-aab7-a8aac4eacc87\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-61f394fa-s7wth" Nov 03 21:58:59.282675 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:59.282572 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2d7sg\" (UniqueName: \"kubernetes.io/projected/5b3c5212-f9b9-4845-aab7-a8aac4eacc87-kube-api-access-2d7sg\") pod \"maas-api-revoke-sub-61f394fa-s7wth\" (UID: \"5b3c5212-f9b9-4845-aab7-a8aac4eacc87\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-61f394fa-s7wth" Nov 03 21:58:59.283874 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:59.283843 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5b3c5212-f9b9-4845-aab7-a8aac4eacc87-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-61f394fa-s7wth\" (UID: \"5b3c5212-f9b9-4845-aab7-a8aac4eacc87\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-61f394fa-s7wth" Nov 03 21:58:59.285670 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:59.285625 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5b3c5212-f9b9-4845-aab7-a8aac4eacc87-maas-api-xks-ca\") pod \"maas-api-revoke-sub-61f394fa-s7wth\" (UID: \"5b3c5212-f9b9-4845-aab7-a8aac4eacc87\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-61f394fa-s7wth" Nov 03 21:58:59.301878 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:59.301816 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2d7sg\" (UniqueName: \"kubernetes.io/projected/5b3c5212-f9b9-4845-aab7-a8aac4eacc87-kube-api-access-2d7sg\") pod \"maas-api-revoke-sub-61f394fa-s7wth\" (UID: \"5b3c5212-f9b9-4845-aab7-a8aac4eacc87\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-61f394fa-s7wth" Nov 03 21:58:59.482233 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:59.481917 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-61f394fa-s7wth" Nov 03 21:58:59.747776 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:58:59.747657 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 21:58:59.873728 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:59.873692 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-61f394fa-s7wth"] Nov 03 21:58:59.876756 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:58:59.876729 6524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b3c5212_f9b9_4845_aab7_a8aac4eacc87.slice/crio-5890ec35fc5fcb6dd2e2170a4e2d7d6b615e6216942e421cb645a03bd1c4330b WatchSource:0}: Error finding container 5890ec35fc5fcb6dd2e2170a4e2d7d6b615e6216942e421cb645a03bd1c4330b: Status 404 returned error can't find the container with id 5890ec35fc5fcb6dd2e2170a4e2d7d6b615e6216942e421cb645a03bd1c4330b Nov 03 21:58:59.976362 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:58:59.976324 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-61f394fa-s7wth" event={"ID":"5b3c5212-f9b9-4845-aab7-a8aac4eacc87","Type":"ContainerStarted","Data":"5890ec35fc5fcb6dd2e2170a4e2d7d6b615e6216942e421cb645a03bd1c4330b"} Nov 03 21:59:00.096502 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:00.096469 6524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" Nov 03 21:59:00.096997 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:00.096972 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" Nov 03 21:59:00.097907 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:00.097886 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" Nov 03 21:59:00.222101 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:00.222067 6524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" Nov 03 21:59:00.222839 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:00.222729 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" Nov 03 21:59:00.223466 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:00.223445 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" Nov 03 21:59:02.001246 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:02.000303 6524 generic.go:356] "Generic (PLEG): container finished" podID="5b3c5212-f9b9-4845-aab7-a8aac4eacc87" containerID="e5d9cd564d3bf6b9ebc0bc35ac55182b0770aa8126790255528520a4b25eb550" exitCode=0 Nov 03 21:59:02.001246 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:02.000440 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-61f394fa-s7wth" event={"ID":"5b3c5212-f9b9-4845-aab7-a8aac4eacc87","Type":"ContainerDied","Data":"e5d9cd564d3bf6b9ebc0bc35ac55182b0770aa8126790255528520a4b25eb550"} Nov 03 21:59:02.019052 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:02.018173 6524 generic.go:356] "Generic (PLEG): container finished" podID="1eabd95e-9b87-42d7-a442-e7cef3a2c73f" containerID="f16ae35446373d360cba236b37ce3fcdf3a3a0bcdd78880b35a7d118161eb0f8" exitCode=0 Nov 03 21:59:02.019052 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:02.018373 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5888686-q6tdc" event={"ID":"1eabd95e-9b87-42d7-a442-e7cef3a2c73f","Type":"ContainerDied","Data":"f16ae35446373d360cba236b37ce3fcdf3a3a0bcdd78880b35a7d118161eb0f8"} Nov 03 21:59:03.166426 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:03.166395 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-61f394fa-s7wth" Nov 03 21:59:03.207073 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:03.207043 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5888686-q6tdc" Nov 03 21:59:03.269303 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:03.269269 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1eabd95e-9b87-42d7-a442-e7cef3a2c73f-kube-api-access-stwn8\" (UniqueName: \"kubernetes.io/projected/1eabd95e-9b87-42d7-a442-e7cef3a2c73f-kube-api-access-stwn8\") pod \"1eabd95e-9b87-42d7-a442-e7cef3a2c73f\" (UID: \"1eabd95e-9b87-42d7-a442-e7cef3a2c73f\") " Nov 03 21:59:03.269507 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:03.269388 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5b3c5212-f9b9-4845-aab7-a8aac4eacc87-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5b3c5212-f9b9-4845-aab7-a8aac4eacc87-maas-api-openshift-ca\") pod \"5b3c5212-f9b9-4845-aab7-a8aac4eacc87\" (UID: \"5b3c5212-f9b9-4845-aab7-a8aac4eacc87\") " Nov 03 21:59:03.269507 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:03.269429 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1eabd95e-9b87-42d7-a442-e7cef3a2c73f-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1eabd95e-9b87-42d7-a442-e7cef3a2c73f-maas-api-openshift-ca\") pod \"1eabd95e-9b87-42d7-a442-e7cef3a2c73f\" (UID: \"1eabd95e-9b87-42d7-a442-e7cef3a2c73f\") " Nov 03 21:59:03.269507 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:03.269468 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5b3c5212-f9b9-4845-aab7-a8aac4eacc87-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5b3c5212-f9b9-4845-aab7-a8aac4eacc87-maas-api-xks-ca\") pod \"5b3c5212-f9b9-4845-aab7-a8aac4eacc87\" (UID: \"5b3c5212-f9b9-4845-aab7-a8aac4eacc87\") " Nov 03 21:59:03.269684 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:03.269526 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5b3c5212-f9b9-4845-aab7-a8aac4eacc87-kube-api-access-2d7sg\" (UniqueName: \"kubernetes.io/projected/5b3c5212-f9b9-4845-aab7-a8aac4eacc87-kube-api-access-2d7sg\") pod \"5b3c5212-f9b9-4845-aab7-a8aac4eacc87\" (UID: \"5b3c5212-f9b9-4845-aab7-a8aac4eacc87\") " Nov 03 21:59:03.269684 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:03.269563 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1eabd95e-9b87-42d7-a442-e7cef3a2c73f-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1eabd95e-9b87-42d7-a442-e7cef3a2c73f-maas-api-xks-ca\") pod \"1eabd95e-9b87-42d7-a442-e7cef3a2c73f\" (UID: \"1eabd95e-9b87-42d7-a442-e7cef3a2c73f\") " Nov 03 21:59:03.269820 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:03.269788 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b3c5212-f9b9-4845-aab7-a8aac4eacc87-maas-api-openshift-ca" pod "5b3c5212-f9b9-4845-aab7-a8aac4eacc87" (UID: "5b3c5212-f9b9-4845-aab7-a8aac4eacc87"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 03 21:59:03.269893 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:03.269860 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1eabd95e-9b87-42d7-a442-e7cef3a2c73f-maas-api-openshift-ca" pod "1eabd95e-9b87-42d7-a442-e7cef3a2c73f" (UID: "1eabd95e-9b87-42d7-a442-e7cef3a2c73f"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 03 21:59:03.270004 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:03.269976 6524 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5b3c5212-f9b9-4845-aab7-a8aac4eacc87-maas-api-openshift-ca\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:03.270004 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:03.270003 6524 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1eabd95e-9b87-42d7-a442-e7cef3a2c73f-maas-api-openshift-ca\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:03.271660 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:03.271627 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1eabd95e-9b87-42d7-a442-e7cef3a2c73f-maas-api-xks-ca" pod "1eabd95e-9b87-42d7-a442-e7cef3a2c73f" (UID: "1eabd95e-9b87-42d7-a442-e7cef3a2c73f"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 03 21:59:03.271771 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:03.271670 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b3c5212-f9b9-4845-aab7-a8aac4eacc87-maas-api-xks-ca" pod "5b3c5212-f9b9-4845-aab7-a8aac4eacc87" (UID: "5b3c5212-f9b9-4845-aab7-a8aac4eacc87"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 03 21:59:03.271771 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:03.271708 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b3c5212-f9b9-4845-aab7-a8aac4eacc87-kube-api-access-2d7sg" pod "5b3c5212-f9b9-4845-aab7-a8aac4eacc87" (UID: "5b3c5212-f9b9-4845-aab7-a8aac4eacc87"). InnerVolumeSpecName "kube-api-access-2d7sg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 03 21:59:03.271771 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:03.271755 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1eabd95e-9b87-42d7-a442-e7cef3a2c73f-kube-api-access-stwn8" pod "1eabd95e-9b87-42d7-a442-e7cef3a2c73f" (UID: "1eabd95e-9b87-42d7-a442-e7cef3a2c73f"). InnerVolumeSpecName "kube-api-access-stwn8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 03 21:59:03.371268 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:03.371225 6524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-stwn8\" (UniqueName: \"kubernetes.io/projected/1eabd95e-9b87-42d7-a442-e7cef3a2c73f-kube-api-access-stwn8\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:03.371268 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:03.371262 6524 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5b3c5212-f9b9-4845-aab7-a8aac4eacc87-maas-api-xks-ca\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:03.371268 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:03.371272 6524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2d7sg\" (UniqueName: \"kubernetes.io/projected/5b3c5212-f9b9-4845-aab7-a8aac4eacc87-kube-api-access-2d7sg\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:03.371500 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:03.371282 6524 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1eabd95e-9b87-42d7-a442-e7cef3a2c73f-maas-api-xks-ca\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:04.028744 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:04.028717 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-61f394fa-s7wth" Nov 03 21:59:04.028744 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:04.028742 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-61f394fa-s7wth" event={"ID":"5b3c5212-f9b9-4845-aab7-a8aac4eacc87","Type":"ContainerDied","Data":"5890ec35fc5fcb6dd2e2170a4e2d7d6b615e6216942e421cb645a03bd1c4330b"} Nov 03 21:59:04.029011 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:04.028784 6524 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5890ec35fc5fcb6dd2e2170a4e2d7d6b615e6216942e421cb645a03bd1c4330b" Nov 03 21:59:04.030014 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:04.029994 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5888686-q6tdc" Nov 03 21:59:04.030116 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:04.030026 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d5888686-q6tdc" event={"ID":"1eabd95e-9b87-42d7-a442-e7cef3a2c73f","Type":"ContainerDied","Data":"3a81b2fcd83d5d8bec5db1ad68e86302e0d11f0da8f0155921b1ae48b5cb8d91"} Nov 03 21:59:04.030116 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:04.030070 6524 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3a81b2fcd83d5d8bec5db1ad68e86302e0d11f0da8f0155921b1ae48b5cb8d91" Nov 03 21:59:06.419104 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.419063 6524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt"] Nov 03 21:59:06.419516 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.419459 6524 cpu_manager.go:414] "containerMap: removing container" podUID="1eabd95e-9b87-42d7-a442-e7cef3a2c73f" containerName="revoke-keys" Nov 03 21:59:06.419516 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.419471 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1eabd95e-9b87-42d7-a442-e7cef3a2c73f" containerName="revoke-keys" Nov 03 21:59:06.419516 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.419499 6524 cpu_manager.go:414] "containerMap: removing container" podUID="5b3c5212-f9b9-4845-aab7-a8aac4eacc87" containerName="revoke-keys" Nov 03 21:59:06.419516 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.419505 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5b3c5212-f9b9-4845-aab7-a8aac4eacc87" containerName="revoke-keys" Nov 03 21:59:06.419643 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.419569 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="5b3c5212-f9b9-4845-aab7-a8aac4eacc87" containerName="revoke-keys" Nov 03 21:59:06.419643 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.419580 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="1eabd95e-9b87-42d7-a442-e7cef3a2c73f" containerName="revoke-keys" Nov 03 21:59:06.423733 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.423715 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" Nov 03 21:59:06.429634 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.429613 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-f4fb1b-openshift-default-dockercfg-pssfw\"" Nov 03 21:59:06.434852 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.434821 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt"] Nov 03 21:59:06.503731 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.503686 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efcc7e46-86ce-427c-a5d7-99878a682ac1-workload-socket\") pod \"e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt\" (UID: \"efcc7e46-86ce-427c-a5d7-99878a682ac1\") " pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" Nov 03 21:59:06.503731 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.503730 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efcc7e46-86ce-427c-a5d7-99878a682ac1-istio-token\") pod \"e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt\" (UID: \"efcc7e46-86ce-427c-a5d7-99878a682ac1\") " pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" Nov 03 21:59:06.503943 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.503850 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvzdg\" (UniqueName: \"kubernetes.io/projected/efcc7e46-86ce-427c-a5d7-99878a682ac1-kube-api-access-hvzdg\") pod \"e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt\" (UID: \"efcc7e46-86ce-427c-a5d7-99878a682ac1\") " pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" Nov 03 21:59:06.503943 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.503913 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efcc7e46-86ce-427c-a5d7-99878a682ac1-istiod-ca-cert\") pod \"e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt\" (UID: \"efcc7e46-86ce-427c-a5d7-99878a682ac1\") " pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" Nov 03 21:59:06.504011 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.503942 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efcc7e46-86ce-427c-a5d7-99878a682ac1-credential-socket\") pod \"e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt\" (UID: \"efcc7e46-86ce-427c-a5d7-99878a682ac1\") " pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" Nov 03 21:59:06.504011 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.503964 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efcc7e46-86ce-427c-a5d7-99878a682ac1-istio-envoy\") pod \"e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt\" (UID: \"efcc7e46-86ce-427c-a5d7-99878a682ac1\") " pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" Nov 03 21:59:06.504011 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.503987 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efcc7e46-86ce-427c-a5d7-99878a682ac1-istio-data\") pod \"e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt\" (UID: \"efcc7e46-86ce-427c-a5d7-99878a682ac1\") " pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" Nov 03 21:59:06.504102 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.504025 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efcc7e46-86ce-427c-a5d7-99878a682ac1-workload-certs\") pod \"e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt\" (UID: \"efcc7e46-86ce-427c-a5d7-99878a682ac1\") " pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" Nov 03 21:59:06.504102 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.504076 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efcc7e46-86ce-427c-a5d7-99878a682ac1-istio-podinfo\") pod \"e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt\" (UID: \"efcc7e46-86ce-427c-a5d7-99878a682ac1\") " pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" Nov 03 21:59:06.569297 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.569263 6524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd"] Nov 03 21:59:06.572929 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.572905 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" Nov 03 21:59:06.577037 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.577013 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-53a2cb-openshift-default-dockercfg-6hq7h\"" Nov 03 21:59:06.586061 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.586030 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd"] Nov 03 21:59:06.605462 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.605422 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d3e9770-476c-47a5-91d7-afffb476b8c7-workload-certs\") pod \"e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd\" (UID: \"1d3e9770-476c-47a5-91d7-afffb476b8c7\") " pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" Nov 03 21:59:06.605841 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.605799 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efcc7e46-86ce-427c-a5d7-99878a682ac1-workload-socket\") pod \"e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt\" (UID: \"efcc7e46-86ce-427c-a5d7-99878a682ac1\") " pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" Nov 03 21:59:06.606222 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.606197 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d3e9770-476c-47a5-91d7-afffb476b8c7-istio-token\") pod \"e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd\" (UID: \"1d3e9770-476c-47a5-91d7-afffb476b8c7\") " pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" Nov 03 21:59:06.606450 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.606418 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efcc7e46-86ce-427c-a5d7-99878a682ac1-istio-token\") pod \"e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt\" (UID: \"efcc7e46-86ce-427c-a5d7-99878a682ac1\") " pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" Nov 03 21:59:06.606548 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.606495 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d3e9770-476c-47a5-91d7-afffb476b8c7-istio-data\") pod \"e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd\" (UID: \"1d3e9770-476c-47a5-91d7-afffb476b8c7\") " pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" Nov 03 21:59:06.606548 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.606535 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvzdg\" (UniqueName: \"kubernetes.io/projected/efcc7e46-86ce-427c-a5d7-99878a682ac1-kube-api-access-hvzdg\") pod \"e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt\" (UID: \"efcc7e46-86ce-427c-a5d7-99878a682ac1\") " pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" Nov 03 21:59:06.606680 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.606639 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwrcl\" (UniqueName: \"kubernetes.io/projected/1d3e9770-476c-47a5-91d7-afffb476b8c7-kube-api-access-fwrcl\") pod \"e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd\" (UID: \"1d3e9770-476c-47a5-91d7-afffb476b8c7\") " pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" Nov 03 21:59:06.606680 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.606671 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efcc7e46-86ce-427c-a5d7-99878a682ac1-istiod-ca-cert\") pod \"e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt\" (UID: \"efcc7e46-86ce-427c-a5d7-99878a682ac1\") " pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" Nov 03 21:59:06.606770 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.606703 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efcc7e46-86ce-427c-a5d7-99878a682ac1-credential-socket\") pod \"e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt\" (UID: \"efcc7e46-86ce-427c-a5d7-99878a682ac1\") " pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" Nov 03 21:59:06.606825 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.606772 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efcc7e46-86ce-427c-a5d7-99878a682ac1-istio-envoy\") pod \"e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt\" (UID: \"efcc7e46-86ce-427c-a5d7-99878a682ac1\") " pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" Nov 03 21:59:06.606825 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.606816 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efcc7e46-86ce-427c-a5d7-99878a682ac1-istio-data\") pod \"e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt\" (UID: \"efcc7e46-86ce-427c-a5d7-99878a682ac1\") " pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" Nov 03 21:59:06.606921 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.606845 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d3e9770-476c-47a5-91d7-afffb476b8c7-istio-podinfo\") pod \"e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd\" (UID: \"1d3e9770-476c-47a5-91d7-afffb476b8c7\") " pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" Nov 03 21:59:06.606921 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.606853 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efcc7e46-86ce-427c-a5d7-99878a682ac1-workload-socket\") pod \"e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt\" (UID: \"efcc7e46-86ce-427c-a5d7-99878a682ac1\") " pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" Nov 03 21:59:06.606921 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.606874 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d3e9770-476c-47a5-91d7-afffb476b8c7-istiod-ca-cert\") pod \"e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd\" (UID: \"1d3e9770-476c-47a5-91d7-afffb476b8c7\") " pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" Nov 03 21:59:06.606921 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.606907 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efcc7e46-86ce-427c-a5d7-99878a682ac1-workload-certs\") pod \"e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt\" (UID: \"efcc7e46-86ce-427c-a5d7-99878a682ac1\") " pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" Nov 03 21:59:06.607154 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.607036 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d3e9770-476c-47a5-91d7-afffb476b8c7-istio-envoy\") pod \"e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd\" (UID: \"1d3e9770-476c-47a5-91d7-afffb476b8c7\") " pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" Nov 03 21:59:06.607154 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.607089 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efcc7e46-86ce-427c-a5d7-99878a682ac1-istio-podinfo\") pod \"e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt\" (UID: \"efcc7e46-86ce-427c-a5d7-99878a682ac1\") " pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" Nov 03 21:59:06.607265 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.607170 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d3e9770-476c-47a5-91d7-afffb476b8c7-workload-socket\") pod \"e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd\" (UID: \"1d3e9770-476c-47a5-91d7-afffb476b8c7\") " pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" Nov 03 21:59:06.607265 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.607199 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d3e9770-476c-47a5-91d7-afffb476b8c7-credential-socket\") pod \"e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd\" (UID: \"1d3e9770-476c-47a5-91d7-afffb476b8c7\") " pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" Nov 03 21:59:06.607265 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.607202 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efcc7e46-86ce-427c-a5d7-99878a682ac1-credential-socket\") pod \"e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt\" (UID: \"efcc7e46-86ce-427c-a5d7-99878a682ac1\") " pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" Nov 03 21:59:06.607265 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.607219 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efcc7e46-86ce-427c-a5d7-99878a682ac1-istio-data\") pod \"e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt\" (UID: \"efcc7e46-86ce-427c-a5d7-99878a682ac1\") " pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" Nov 03 21:59:06.607464 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.607370 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efcc7e46-86ce-427c-a5d7-99878a682ac1-workload-certs\") pod \"e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt\" (UID: \"efcc7e46-86ce-427c-a5d7-99878a682ac1\") " pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" Nov 03 21:59:06.607840 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.607819 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efcc7e46-86ce-427c-a5d7-99878a682ac1-istiod-ca-cert\") pod \"e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt\" (UID: \"efcc7e46-86ce-427c-a5d7-99878a682ac1\") " pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" Nov 03 21:59:06.609990 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.609962 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efcc7e46-86ce-427c-a5d7-99878a682ac1-istio-envoy\") pod \"e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt\" (UID: \"efcc7e46-86ce-427c-a5d7-99878a682ac1\") " pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" Nov 03 21:59:06.610233 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.610214 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efcc7e46-86ce-427c-a5d7-99878a682ac1-istio-podinfo\") pod \"e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt\" (UID: \"efcc7e46-86ce-427c-a5d7-99878a682ac1\") " pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" Nov 03 21:59:06.619375 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.619342 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efcc7e46-86ce-427c-a5d7-99878a682ac1-istio-token\") pod \"e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt\" (UID: \"efcc7e46-86ce-427c-a5d7-99878a682ac1\") " pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" Nov 03 21:59:06.619661 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.619637 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvzdg\" (UniqueName: \"kubernetes.io/projected/efcc7e46-86ce-427c-a5d7-99878a682ac1-kube-api-access-hvzdg\") pod \"e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt\" (UID: \"efcc7e46-86ce-427c-a5d7-99878a682ac1\") " pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" Nov 03 21:59:06.694116 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.694036 6524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45"] Nov 03 21:59:06.699265 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.699236 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" Nov 03 21:59:06.706662 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.706634 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-fc16de-openshift-default-dockercfg-thxpt\"" Nov 03 21:59:06.708080 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.708042 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d3e9770-476c-47a5-91d7-afffb476b8c7-workload-socket\") pod \"e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd\" (UID: \"1d3e9770-476c-47a5-91d7-afffb476b8c7\") " pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" Nov 03 21:59:06.708220 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.708095 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d3e9770-476c-47a5-91d7-afffb476b8c7-credential-socket\") pod \"e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd\" (UID: \"1d3e9770-476c-47a5-91d7-afffb476b8c7\") " pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" Nov 03 21:59:06.708220 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.708167 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d3e9770-476c-47a5-91d7-afffb476b8c7-workload-certs\") pod \"e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd\" (UID: \"1d3e9770-476c-47a5-91d7-afffb476b8c7\") " pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" Nov 03 21:59:06.708220 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.708200 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d3e9770-476c-47a5-91d7-afffb476b8c7-istio-token\") pod \"e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd\" (UID: \"1d3e9770-476c-47a5-91d7-afffb476b8c7\") " pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" Nov 03 21:59:06.708448 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.708261 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d3e9770-476c-47a5-91d7-afffb476b8c7-istio-data\") pod \"e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd\" (UID: \"1d3e9770-476c-47a5-91d7-afffb476b8c7\") " pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" Nov 03 21:59:06.708448 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.708351 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwrcl\" (UniqueName: \"kubernetes.io/projected/1d3e9770-476c-47a5-91d7-afffb476b8c7-kube-api-access-fwrcl\") pod \"e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd\" (UID: \"1d3e9770-476c-47a5-91d7-afffb476b8c7\") " pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" Nov 03 21:59:06.708551 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.708468 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d3e9770-476c-47a5-91d7-afffb476b8c7-workload-socket\") pod \"e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd\" (UID: \"1d3e9770-476c-47a5-91d7-afffb476b8c7\") " pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" Nov 03 21:59:06.708551 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.708520 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d3e9770-476c-47a5-91d7-afffb476b8c7-istio-podinfo\") pod \"e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd\" (UID: \"1d3e9770-476c-47a5-91d7-afffb476b8c7\") " pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" Nov 03 21:59:06.708653 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.708550 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d3e9770-476c-47a5-91d7-afffb476b8c7-istiod-ca-cert\") pod \"e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd\" (UID: \"1d3e9770-476c-47a5-91d7-afffb476b8c7\") " pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" Nov 03 21:59:06.708653 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.708588 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d3e9770-476c-47a5-91d7-afffb476b8c7-istio-envoy\") pod \"e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd\" (UID: \"1d3e9770-476c-47a5-91d7-afffb476b8c7\") " pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" Nov 03 21:59:06.708653 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.708520 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d3e9770-476c-47a5-91d7-afffb476b8c7-credential-socket\") pod \"e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd\" (UID: \"1d3e9770-476c-47a5-91d7-afffb476b8c7\") " pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" Nov 03 21:59:06.708806 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.708587 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d3e9770-476c-47a5-91d7-afffb476b8c7-workload-certs\") pod \"e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd\" (UID: \"1d3e9770-476c-47a5-91d7-afffb476b8c7\") " pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" Nov 03 21:59:06.708806 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.708700 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d3e9770-476c-47a5-91d7-afffb476b8c7-istio-data\") pod \"e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd\" (UID: \"1d3e9770-476c-47a5-91d7-afffb476b8c7\") " pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" Nov 03 21:59:06.709282 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.709257 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d3e9770-476c-47a5-91d7-afffb476b8c7-istiod-ca-cert\") pod \"e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd\" (UID: \"1d3e9770-476c-47a5-91d7-afffb476b8c7\") " pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" Nov 03 21:59:06.710808 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.710786 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d3e9770-476c-47a5-91d7-afffb476b8c7-istio-envoy\") pod \"e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd\" (UID: \"1d3e9770-476c-47a5-91d7-afffb476b8c7\") " pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" Nov 03 21:59:06.711114 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.711093 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d3e9770-476c-47a5-91d7-afffb476b8c7-istio-podinfo\") pod \"e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd\" (UID: \"1d3e9770-476c-47a5-91d7-afffb476b8c7\") " pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" Nov 03 21:59:06.715216 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.715165 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45"] Nov 03 21:59:06.730522 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.726113 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwrcl\" (UniqueName: \"kubernetes.io/projected/1d3e9770-476c-47a5-91d7-afffb476b8c7-kube-api-access-fwrcl\") pod \"e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd\" (UID: \"1d3e9770-476c-47a5-91d7-afffb476b8c7\") " pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" Nov 03 21:59:06.730522 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.728277 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d3e9770-476c-47a5-91d7-afffb476b8c7-istio-token\") pod \"e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd\" (UID: \"1d3e9770-476c-47a5-91d7-afffb476b8c7\") " pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" Nov 03 21:59:06.738720 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.737956 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" Nov 03 21:59:06.809828 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.809790 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-istio-token\") pod \"e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45\" (UID: \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\") " pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" Nov 03 21:59:06.809977 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.809845 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-workload-socket\") pod \"e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45\" (UID: \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\") " pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" Nov 03 21:59:06.809977 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.809882 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-workload-certs\") pod \"e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45\" (UID: \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\") " pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" Nov 03 21:59:06.809977 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.809920 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-istiod-ca-cert\") pod \"e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45\" (UID: \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\") " pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" Nov 03 21:59:06.809977 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.809951 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-credential-socket\") pod \"e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45\" (UID: \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\") " pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" Nov 03 21:59:06.809977 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.809965 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-istio-envoy\") pod \"e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45\" (UID: \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\") " pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" Nov 03 21:59:06.810148 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.809997 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvv9x\" (UniqueName: \"kubernetes.io/projected/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-kube-api-access-qvv9x\") pod \"e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45\" (UID: \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\") " pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" Nov 03 21:59:06.810148 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.810016 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-istio-data\") pod \"e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45\" (UID: \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\") " pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" Nov 03 21:59:06.810148 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.810032 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-istio-podinfo\") pod \"e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45\" (UID: \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\") " pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" Nov 03 21:59:06.888558 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.887315 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" Nov 03 21:59:06.911134 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.911090 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-istiod-ca-cert\") pod \"e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45\" (UID: \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\") " pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" Nov 03 21:59:06.911287 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.911167 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-credential-socket\") pod \"e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45\" (UID: \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\") " pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" Nov 03 21:59:06.911287 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.911190 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-istio-envoy\") pod \"e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45\" (UID: \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\") " pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" Nov 03 21:59:06.911287 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.911233 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvv9x\" (UniqueName: \"kubernetes.io/projected/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-kube-api-access-qvv9x\") pod \"e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45\" (UID: \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\") " pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" Nov 03 21:59:06.911287 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.911264 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-istio-data\") pod \"e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45\" (UID: \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\") " pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" Nov 03 21:59:06.911498 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.911289 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-istio-podinfo\") pod \"e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45\" (UID: \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\") " pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" Nov 03 21:59:06.911498 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.911358 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-istio-token\") pod \"e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45\" (UID: \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\") " pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" Nov 03 21:59:06.911498 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.911402 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-workload-socket\") pod \"e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45\" (UID: \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\") " pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" Nov 03 21:59:06.911498 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.911459 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-workload-certs\") pod \"e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45\" (UID: \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\") " pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" Nov 03 21:59:06.911692 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.911595 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-credential-socket\") pod \"e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45\" (UID: \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\") " pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" Nov 03 21:59:06.911945 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.911921 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-istiod-ca-cert\") pod \"e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45\" (UID: \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\") " pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" Nov 03 21:59:06.912031 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.911939 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-workload-socket\") pod \"e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45\" (UID: \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\") " pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" Nov 03 21:59:06.912031 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.911921 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-istio-data\") pod \"e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45\" (UID: \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\") " pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" Nov 03 21:59:06.912031 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.911902 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-workload-certs\") pod \"e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45\" (UID: \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\") " pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" Nov 03 21:59:06.914589 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.914238 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-istio-podinfo\") pod \"e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45\" (UID: \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\") " pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" Nov 03 21:59:06.914589 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.914538 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-istio-envoy\") pod \"e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45\" (UID: \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\") " pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" Nov 03 21:59:06.925491 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.925452 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-istio-token\") pod \"e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45\" (UID: \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\") " pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" Nov 03 21:59:06.925941 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.925919 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvv9x\" (UniqueName: \"kubernetes.io/projected/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-kube-api-access-qvv9x\") pod \"e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45\" (UID: \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\") " pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" Nov 03 21:59:06.943814 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.943472 6524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9"] Nov 03 21:59:06.949165 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.948582 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" Nov 03 21:59:06.952556 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.952474 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-8af3f3-openshift-default-dockercfg-hsqhd\"" Nov 03 21:59:06.969312 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:06.964655 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9"] Nov 03 21:59:07.012448 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.012414 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/779feacc-19e1-4b9e-9968-896c178c78f8-workload-certs\") pod \"e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9\" (UID: \"779feacc-19e1-4b9e-9968-896c178c78f8\") " pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" Nov 03 21:59:07.012612 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.012464 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/779feacc-19e1-4b9e-9968-896c178c78f8-credential-socket\") pod \"e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9\" (UID: \"779feacc-19e1-4b9e-9968-896c178c78f8\") " pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" Nov 03 21:59:07.012612 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.012545 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/779feacc-19e1-4b9e-9968-896c178c78f8-istio-envoy\") pod \"e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9\" (UID: \"779feacc-19e1-4b9e-9968-896c178c78f8\") " pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" Nov 03 21:59:07.012612 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.012586 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/779feacc-19e1-4b9e-9968-896c178c78f8-istio-podinfo\") pod \"e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9\" (UID: \"779feacc-19e1-4b9e-9968-896c178c78f8\") " pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" Nov 03 21:59:07.012713 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.012660 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/779feacc-19e1-4b9e-9968-896c178c78f8-istio-data\") pod \"e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9\" (UID: \"779feacc-19e1-4b9e-9968-896c178c78f8\") " pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" Nov 03 21:59:07.012713 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.012693 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/779feacc-19e1-4b9e-9968-896c178c78f8-workload-socket\") pod \"e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9\" (UID: \"779feacc-19e1-4b9e-9968-896c178c78f8\") " pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" Nov 03 21:59:07.012778 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.012766 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/779feacc-19e1-4b9e-9968-896c178c78f8-istiod-ca-cert\") pod \"e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9\" (UID: \"779feacc-19e1-4b9e-9968-896c178c78f8\") " pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" Nov 03 21:59:07.012834 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.012823 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5b9w\" (UniqueName: \"kubernetes.io/projected/779feacc-19e1-4b9e-9968-896c178c78f8-kube-api-access-f5b9w\") pod \"e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9\" (UID: \"779feacc-19e1-4b9e-9968-896c178c78f8\") " pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" Nov 03 21:59:07.012869 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.012848 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/779feacc-19e1-4b9e-9968-896c178c78f8-istio-token\") pod \"e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9\" (UID: \"779feacc-19e1-4b9e-9968-896c178c78f8\") " pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" Nov 03 21:59:07.014191 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.014170 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" Nov 03 21:59:07.027902 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.021195 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt"] Nov 03 21:59:07.030116 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.030074 6524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"} Nov 03 21:59:07.030231 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.030179 6524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"} Nov 03 21:59:07.030231 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.030221 6524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"} Nov 03 21:59:07.045471 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.045433 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" event={"ID":"efcc7e46-86ce-427c-a5d7-99878a682ac1","Type":"ContainerStarted","Data":"3df408bef998235765aeb3e9e4f1953244b38be3bf851d417210bfd186ce5454"} Nov 03 21:59:07.114413 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.114380 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/779feacc-19e1-4b9e-9968-896c178c78f8-workload-certs\") pod \"e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9\" (UID: \"779feacc-19e1-4b9e-9968-896c178c78f8\") " pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" Nov 03 21:59:07.114545 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.114432 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/779feacc-19e1-4b9e-9968-896c178c78f8-credential-socket\") pod \"e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9\" (UID: \"779feacc-19e1-4b9e-9968-896c178c78f8\") " pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" Nov 03 21:59:07.114545 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.114448 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/779feacc-19e1-4b9e-9968-896c178c78f8-istio-envoy\") pod \"e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9\" (UID: \"779feacc-19e1-4b9e-9968-896c178c78f8\") " pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" Nov 03 21:59:07.114545 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.114468 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/779feacc-19e1-4b9e-9968-896c178c78f8-istio-podinfo\") pod \"e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9\" (UID: \"779feacc-19e1-4b9e-9968-896c178c78f8\") " pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" Nov 03 21:59:07.114545 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.114508 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/779feacc-19e1-4b9e-9968-896c178c78f8-istio-data\") pod \"e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9\" (UID: \"779feacc-19e1-4b9e-9968-896c178c78f8\") " pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" Nov 03 21:59:07.114545 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.114536 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/779feacc-19e1-4b9e-9968-896c178c78f8-workload-socket\") pod \"e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9\" (UID: \"779feacc-19e1-4b9e-9968-896c178c78f8\") " pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" Nov 03 21:59:07.114708 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.114582 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/779feacc-19e1-4b9e-9968-896c178c78f8-istiod-ca-cert\") pod \"e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9\" (UID: \"779feacc-19e1-4b9e-9968-896c178c78f8\") " pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" Nov 03 21:59:07.114708 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.114613 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5b9w\" (UniqueName: \"kubernetes.io/projected/779feacc-19e1-4b9e-9968-896c178c78f8-kube-api-access-f5b9w\") pod \"e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9\" (UID: \"779feacc-19e1-4b9e-9968-896c178c78f8\") " pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" Nov 03 21:59:07.114708 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.114647 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/779feacc-19e1-4b9e-9968-896c178c78f8-istio-token\") pod \"e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9\" (UID: \"779feacc-19e1-4b9e-9968-896c178c78f8\") " pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" Nov 03 21:59:07.114857 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.114828 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/779feacc-19e1-4b9e-9968-896c178c78f8-workload-certs\") pod \"e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9\" (UID: \"779feacc-19e1-4b9e-9968-896c178c78f8\") " pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" Nov 03 21:59:07.115150 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.115112 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/779feacc-19e1-4b9e-9968-896c178c78f8-istio-data\") pod \"e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9\" (UID: \"779feacc-19e1-4b9e-9968-896c178c78f8\") " pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" Nov 03 21:59:07.115201 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.115185 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/779feacc-19e1-4b9e-9968-896c178c78f8-workload-socket\") pod \"e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9\" (UID: \"779feacc-19e1-4b9e-9968-896c178c78f8\") " pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" Nov 03 21:59:07.115413 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.115393 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/779feacc-19e1-4b9e-9968-896c178c78f8-credential-socket\") pod \"e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9\" (UID: \"779feacc-19e1-4b9e-9968-896c178c78f8\") " pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" Nov 03 21:59:07.115741 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.115722 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/779feacc-19e1-4b9e-9968-896c178c78f8-istiod-ca-cert\") pod \"e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9\" (UID: \"779feacc-19e1-4b9e-9968-896c178c78f8\") " pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" Nov 03 21:59:07.117466 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.117429 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/779feacc-19e1-4b9e-9968-896c178c78f8-istio-envoy\") pod \"e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9\" (UID: \"779feacc-19e1-4b9e-9968-896c178c78f8\") " pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" Nov 03 21:59:07.118098 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.118076 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/779feacc-19e1-4b9e-9968-896c178c78f8-istio-podinfo\") pod \"e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9\" (UID: \"779feacc-19e1-4b9e-9968-896c178c78f8\") " pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" Nov 03 21:59:07.126968 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.126939 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/779feacc-19e1-4b9e-9968-896c178c78f8-istio-token\") pod \"e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9\" (UID: \"779feacc-19e1-4b9e-9968-896c178c78f8\") " pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" Nov 03 21:59:07.128527 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.128473 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5b9w\" (UniqueName: \"kubernetes.io/projected/779feacc-19e1-4b9e-9968-896c178c78f8-kube-api-access-f5b9w\") pod \"e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9\" (UID: \"779feacc-19e1-4b9e-9968-896c178c78f8\") " pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" Nov 03 21:59:07.171255 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.171223 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd"] Nov 03 21:59:07.173837 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:59:07.173804 6524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d3e9770_476c_47a5_91d7_afffb476b8c7.slice/crio-9dcf0a782c1b39a9474c1cada5a18f538460be400910cae40f265cf020291245 WatchSource:0}: Error finding container 9dcf0a782c1b39a9474c1cada5a18f538460be400910cae40f265cf020291245: Status 404 returned error can't find the container with id 9dcf0a782c1b39a9474c1cada5a18f538460be400910cae40f265cf020291245 Nov 03 21:59:07.176787 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.176744 6524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"} Nov 03 21:59:07.176883 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.176846 6524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"} Nov 03 21:59:07.176957 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.176887 6524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"} Nov 03 21:59:07.271167 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.271089 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" Nov 03 21:59:07.306607 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.306573 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45"] Nov 03 21:59:07.310346 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:59:07.310320 6524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9593fbc_9af2_4c1f_a986_3dc48dab11a7.slice/crio-f77c00b7ba1e9c2ae7e80a2267b0d83c3bd9df74329d22eb673d8681b0320efa WatchSource:0}: Error finding container f77c00b7ba1e9c2ae7e80a2267b0d83c3bd9df74329d22eb673d8681b0320efa: Status 404 returned error can't find the container with id f77c00b7ba1e9c2ae7e80a2267b0d83c3bd9df74329d22eb673d8681b0320efa Nov 03 21:59:07.312599 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.312566 6524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"} Nov 03 21:59:07.312705 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.312634 6524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"} Nov 03 21:59:07.312705 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.312663 6524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"} Nov 03 21:59:07.581003 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.578039 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9"] Nov 03 21:59:07.583435 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:59:07.582762 6524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod779feacc_19e1_4b9e_9968_896c178c78f8.slice/crio-1eac504f6610f08b657baab298c3fbea604b050a7d7ce872570634afabb25c63 WatchSource:0}: Error finding container 1eac504f6610f08b657baab298c3fbea604b050a7d7ce872570634afabb25c63: Status 404 returned error can't find the container with id 1eac504f6610f08b657baab298c3fbea604b050a7d7ce872570634afabb25c63 Nov 03 21:59:07.591152 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.589991 6524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"} Nov 03 21:59:07.591152 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.590105 6524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"} Nov 03 21:59:07.591152 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:07.590183 6524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"} Nov 03 21:59:08.069216 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:08.068308 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" event={"ID":"a9593fbc-9af2-4c1f-a986-3dc48dab11a7","Type":"ContainerStarted","Data":"99e28e6f8cf025b81892cd5b47a26086c85b17189fb870aa1076caf8f3b98988"} Nov 03 21:59:08.069216 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:08.068377 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" event={"ID":"a9593fbc-9af2-4c1f-a986-3dc48dab11a7","Type":"ContainerStarted","Data":"f77c00b7ba1e9c2ae7e80a2267b0d83c3bd9df74329d22eb673d8681b0320efa"} Nov 03 21:59:08.072965 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:08.072927 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" event={"ID":"efcc7e46-86ce-427c-a5d7-99878a682ac1","Type":"ContainerStarted","Data":"f00351cebd65d645d82b2ea0a3592d9e03a611ebfe205b7c9edffc54c124aaec"} Nov 03 21:59:08.076863 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:08.076830 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" event={"ID":"779feacc-19e1-4b9e-9968-896c178c78f8","Type":"ContainerStarted","Data":"4a48814aeb26f05ca436d6300a258f42e728939c47bb85f7fbb0ae1fafa98bce"} Nov 03 21:59:08.077018 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:08.076885 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" event={"ID":"779feacc-19e1-4b9e-9968-896c178c78f8","Type":"ContainerStarted","Data":"1eac504f6610f08b657baab298c3fbea604b050a7d7ce872570634afabb25c63"} Nov 03 21:59:08.078702 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:08.078674 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" event={"ID":"1d3e9770-476c-47a5-91d7-afffb476b8c7","Type":"ContainerStarted","Data":"9c2ab59138d7016737610ce46a8b1def9de74b3e058362dcacd9fab67f6178a9"} Nov 03 21:59:08.078828 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:08.078724 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" event={"ID":"1d3e9770-476c-47a5-91d7-afffb476b8c7","Type":"ContainerStarted","Data":"9dcf0a782c1b39a9474c1cada5a18f538460be400910cae40f265cf020291245"} Nov 03 21:59:08.095876 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:08.095808 6524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" podStartSLOduration=2.095789095 podStartE2EDuration="2.095789095s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:59:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:59:08.094417893 +0000 UTC m=+793.446744688" watchObservedRunningTime="2026-11-03 21:59:08.095789095 +0000 UTC m=+793.448115899" Nov 03 21:59:08.163258 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:08.163195 6524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" podStartSLOduration=2.163175331 podStartE2EDuration="2.163175331s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:59:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:59:08.129762402 +0000 UTC m=+793.482089219" watchObservedRunningTime="2026-11-03 21:59:08.163175331 +0000 UTC m=+793.515502135" Nov 03 21:59:08.209148 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:08.208151 6524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" podStartSLOduration=2.208113851 podStartE2EDuration="2.208113851s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:59:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:59:08.171298055 +0000 UTC m=+793.523624860" watchObservedRunningTime="2026-11-03 21:59:08.208113851 +0000 UTC m=+793.560440655" Nov 03 21:59:08.213147 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:08.210392 6524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" podStartSLOduration=2.210373155 podStartE2EDuration="2.210373155s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:59:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:59:08.207641983 +0000 UTC m=+793.559968801" watchObservedRunningTime="2026-11-03 21:59:08.210373155 +0000 UTC m=+793.562699960" Nov 03 21:59:08.285637 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:08.285601 6524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" Nov 03 21:59:08.285977 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:08.285953 6524 patch_prober.go:29] interesting pod/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Nov 03 21:59:08.286137 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:08.286095 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" podUID="779feacc-19e1-4b9e-9968-896c178c78f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Nov 03 21:59:08.740148 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:08.739570 6524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" Nov 03 21:59:08.741977 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:08.741944 6524 patch_prober.go:29] interesting pod/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Nov 03 21:59:08.742224 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:08.742195 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" podUID="efcc7e46-86ce-427c-a5d7-99878a682ac1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Nov 03 21:59:08.892150 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:08.891573 6524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" Nov 03 21:59:08.902149 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:08.899291 6524 patch_prober.go:29] interesting pod/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Nov 03 21:59:08.902149 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:08.899374 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" podUID="1d3e9770-476c-47a5-91d7-afffb476b8c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Nov 03 21:59:09.018206 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:09.017295 6524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" Nov 03 21:59:09.019523 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:09.019485 6524 patch_prober.go:29] interesting pod/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Nov 03 21:59:09.019735 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:09.019695 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" podUID="a9593fbc-9af2-4c1f-a986-3dc48dab11a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Nov 03 21:59:09.309612 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:09.309508 6524 patch_prober.go:29] interesting pod/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Nov 03 21:59:09.319562 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:09.319513 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" podUID="779feacc-19e1-4b9e-9968-896c178c78f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Nov 03 21:59:09.741646 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:09.741598 6524 patch_prober.go:29] interesting pod/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Nov 03 21:59:09.742299 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:09.742262 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" podUID="efcc7e46-86ce-427c-a5d7-99878a682ac1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Nov 03 21:59:09.900171 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:09.896600 6524 patch_prober.go:29] interesting pod/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Nov 03 21:59:09.900171 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:09.896681 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" podUID="1d3e9770-476c-47a5-91d7-afffb476b8c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Nov 03 21:59:10.021195 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:10.014883 6524 patch_prober.go:29] interesting pod/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Nov 03 21:59:10.021195 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:10.014969 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" podUID="a9593fbc-9af2-4c1f-a986-3dc48dab11a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Nov 03 21:59:10.279208 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:10.276324 6524 patch_prober.go:29] interesting pod/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Nov 03 21:59:10.279208 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:10.276406 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" podUID="779feacc-19e1-4b9e-9968-896c178c78f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Nov 03 21:59:10.739489 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:10.739450 6524 patch_prober.go:29] interesting pod/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Nov 03 21:59:10.739776 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:10.739747 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" podUID="efcc7e46-86ce-427c-a5d7-99878a682ac1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Nov 03 21:59:10.892149 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:10.888527 6524 patch_prober.go:29] interesting pod/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Nov 03 21:59:10.892149 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:10.888590 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" podUID="1d3e9770-476c-47a5-91d7-afffb476b8c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Nov 03 21:59:11.029210 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:11.024568 6524 patch_prober.go:29] interesting pod/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Nov 03 21:59:11.029210 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:11.024632 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" podUID="a9593fbc-9af2-4c1f-a986-3dc48dab11a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Nov 03 21:59:11.357150 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:11.356805 6524 patch_prober.go:29] interesting pod/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Nov 03 21:59:11.357150 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:11.356874 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" podUID="779feacc-19e1-4b9e-9968-896c178c78f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Nov 03 21:59:11.721338 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:11.721222 6524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf" podUID="1ab01063-2737-4b29-ade3-ef9cc00ab5e1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.32:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Nov 03 21:59:11.746282 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:11.746245 6524 patch_prober.go:29] interesting pod/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Nov 03 21:59:11.746558 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:11.746531 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" podUID="efcc7e46-86ce-427c-a5d7-99878a682ac1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Nov 03 21:59:11.759162 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:11.747424 6524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-50e65b6f-k2hvk"] Nov 03 21:59:11.812546 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:11.812510 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-50e65b6f-k2hvk"] Nov 03 21:59:11.812960 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:11.812939 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-50e65b6f-k2hvk" Nov 03 21:59:11.858530 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:11.858494 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Nov 03 21:59:11.869316 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:11.869270 6524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j" podUID="7df4ae2c-a36f-4933-9e10-1f712fb47595" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Nov 03 21:59:11.885173 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:11.875543 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5n7j7\"" Nov 03 21:59:11.885173 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:11.876218 6524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z" podUID="e0ef1acd-e793-4ea1-b89c-924d84be20bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.33:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Nov 03 21:59:11.885173 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:11.880227 6524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv" podUID="851596d1-02e4-42b9-bdab-19a9321a2a04" containerName="main" probeResult="failure" output="Get \"https://10.133.0.31:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Nov 03 21:59:11.916661 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:11.902260 6524 patch_prober.go:29] interesting pod/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Nov 03 21:59:11.916661 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:11.902323 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" podUID="1d3e9770-476c-47a5-91d7-afffb476b8c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Nov 03 21:59:12.008152 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:12.000630 6524 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z" podUID="e0ef1acd-e793-4ea1-b89c-924d84be20bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.33:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Nov 03 21:59:12.032150 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:12.028225 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnvzw\" (UniqueName: \"kubernetes.io/projected/4e9aba93-9243-44df-94e7-33ca432c011b-kube-api-access-bnvzw\") pod \"maas-api-revoke-sub-50e65b6f-k2hvk\" (UID: \"4e9aba93-9243-44df-94e7-33ca432c011b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-50e65b6f-k2hvk" Nov 03 21:59:12.032150 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:12.028446 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4e9aba93-9243-44df-94e7-33ca432c011b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-50e65b6f-k2hvk\" (UID: \"4e9aba93-9243-44df-94e7-33ca432c011b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-50e65b6f-k2hvk" Nov 03 21:59:12.032150 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:12.028527 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4e9aba93-9243-44df-94e7-33ca432c011b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-50e65b6f-k2hvk\" (UID: \"4e9aba93-9243-44df-94e7-33ca432c011b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-50e65b6f-k2hvk" Nov 03 21:59:12.134152 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:12.131595 6524 patch_prober.go:29] interesting pod/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Nov 03 21:59:12.134152 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:12.131662 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" podUID="a9593fbc-9af2-4c1f-a986-3dc48dab11a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Nov 03 21:59:12.136980 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:12.136938 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4e9aba93-9243-44df-94e7-33ca432c011b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-50e65b6f-k2hvk\" (UID: \"4e9aba93-9243-44df-94e7-33ca432c011b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-50e65b6f-k2hvk" Nov 03 21:59:12.137729 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:12.137702 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4e9aba93-9243-44df-94e7-33ca432c011b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-50e65b6f-k2hvk\" (UID: \"4e9aba93-9243-44df-94e7-33ca432c011b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-50e65b6f-k2hvk" Nov 03 21:59:12.186147 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:12.138531 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4e9aba93-9243-44df-94e7-33ca432c011b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-50e65b6f-k2hvk\" (UID: \"4e9aba93-9243-44df-94e7-33ca432c011b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-50e65b6f-k2hvk" Nov 03 21:59:12.186147 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:12.185428 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnvzw\" (UniqueName: \"kubernetes.io/projected/4e9aba93-9243-44df-94e7-33ca432c011b-kube-api-access-bnvzw\") pod \"maas-api-revoke-sub-50e65b6f-k2hvk\" (UID: \"4e9aba93-9243-44df-94e7-33ca432c011b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-50e65b6f-k2hvk" Nov 03 21:59:12.269487 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:12.253280 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4e9aba93-9243-44df-94e7-33ca432c011b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-50e65b6f-k2hvk\" (UID: \"4e9aba93-9243-44df-94e7-33ca432c011b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-50e65b6f-k2hvk" Nov 03 21:59:12.269487 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:12.269267 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnvzw\" (UniqueName: \"kubernetes.io/projected/4e9aba93-9243-44df-94e7-33ca432c011b-kube-api-access-bnvzw\") pod \"maas-api-revoke-sub-50e65b6f-k2hvk\" (UID: \"4e9aba93-9243-44df-94e7-33ca432c011b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-50e65b6f-k2hvk" Nov 03 21:59:12.275583 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:12.275546 6524 patch_prober.go:29] interesting pod/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Nov 03 21:59:12.275836 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:12.275806 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" podUID="779feacc-19e1-4b9e-9968-896c178c78f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Nov 03 21:59:12.357576 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:12.357452 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-50e65b6f-k2hvk" Nov 03 21:59:12.735272 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:12.735218 6524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf" podUID="1ab01063-2737-4b29-ade3-ef9cc00ab5e1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.32:8000/ready\": context deadline exceeded" Nov 03 21:59:12.761438 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:12.759265 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf" Nov 03 21:59:12.849153 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:12.848251 6524 patch_prober.go:29] interesting pod/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Nov 03 21:59:12.849153 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:12.848325 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" podUID="efcc7e46-86ce-427c-a5d7-99878a682ac1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Nov 03 21:59:12.873199 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:12.872531 6524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j" podUID="7df4ae2c-a36f-4933-9e10-1f712fb47595" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": context deadline exceeded" Nov 03 21:59:12.877946 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:12.877208 6524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z" podUID="e0ef1acd-e793-4ea1-b89c-924d84be20bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.33:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Nov 03 21:59:12.904733 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:12.904674 6524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv" podUID="851596d1-02e4-42b9-bdab-19a9321a2a04" containerName="main" probeResult="failure" output="Get \"https://10.133.0.31:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Nov 03 21:59:12.921150 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:12.906368 6524 patch_prober.go:29] interesting pod/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Nov 03 21:59:12.922099 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:12.922049 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" podUID="1d3e9770-476c-47a5-91d7-afffb476b8c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Nov 03 21:59:12.964165 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:12.937762 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j" Nov 03 21:59:12.964165 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:12.937859 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z" Nov 03 21:59:12.964165 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:12.937901 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf" Nov 03 21:59:12.964165 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:12.937945 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv" Nov 03 21:59:13.019252 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:13.019055 6524 patch_prober.go:29] interesting pod/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Nov 03 21:59:13.019252 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:13.019113 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" podUID="a9593fbc-9af2-4c1f-a986-3dc48dab11a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Nov 03 21:59:13.183812 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:13.183783 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv" Nov 03 21:59:13.312208 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:13.307536 6524 patch_prober.go:29] interesting pod/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Nov 03 21:59:13.312510 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:13.312463 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" podUID="779feacc-19e1-4b9e-9968-896c178c78f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Nov 03 21:59:13.441153 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:13.406512 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j" Nov 03 21:59:13.758394 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:13.742447 6524 patch_prober.go:29] interesting pod/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Nov 03 21:59:13.758394 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:13.742510 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" podUID="efcc7e46-86ce-427c-a5d7-99878a682ac1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Nov 03 21:59:13.878737 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:13.878697 6524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z" podUID="e0ef1acd-e793-4ea1-b89c-924d84be20bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.33:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Nov 03 21:59:13.909356 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:13.904288 6524 patch_prober.go:29] interesting pod/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Nov 03 21:59:13.909356 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:13.904350 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" podUID="1d3e9770-476c-47a5-91d7-afffb476b8c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Nov 03 21:59:14.055221 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:14.004983 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-50e65b6f-k2hvk"] Nov 03 21:59:14.099249 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:14.099216 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z" Nov 03 21:59:14.100367 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:14.049050 6524 patch_prober.go:29] interesting pod/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Nov 03 21:59:14.100367 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:14.099751 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" podUID="a9593fbc-9af2-4c1f-a986-3dc48dab11a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Nov 03 21:59:14.201151 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:14.187101 6524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv" podUID="851596d1-02e4-42b9-bdab-19a9321a2a04" containerName="main" probeResult="failure" output="Get \"https://10.133.0.31:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Nov 03 21:59:14.283150 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:14.279921 6524 patch_prober.go:29] interesting pod/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Nov 03 21:59:14.283150 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:14.279984 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" podUID="779feacc-19e1-4b9e-9968-896c178c78f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Nov 03 21:59:14.522765 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:14.522717 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-50e65b6f-k2hvk" event={"ID":"4e9aba93-9243-44df-94e7-33ca432c011b","Type":"ContainerStarted","Data":"94fe36687786f7a836b046b4336eb96838100c47678b7985e5bfa06c41d602fb"} Nov 03 21:59:14.741206 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:14.740548 6524 patch_prober.go:29] interesting pod/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Nov 03 21:59:14.741206 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:14.740609 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" podUID="efcc7e46-86ce-427c-a5d7-99878a682ac1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Nov 03 21:59:14.888549 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:14.888509 6524 patch_prober.go:29] interesting pod/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Nov 03 21:59:14.941261 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:14.941210 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" podUID="1d3e9770-476c-47a5-91d7-afffb476b8c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Nov 03 21:59:14.943023 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:14.920447 6524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf" podUID="1ab01063-2737-4b29-ade3-ef9cc00ab5e1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.32:8000/ready\": context deadline exceeded" Nov 03 21:59:15.015625 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:15.015522 6524 patch_prober.go:29] interesting pod/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Nov 03 21:59:15.015865 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:15.015834 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" podUID="a9593fbc-9af2-4c1f-a986-3dc48dab11a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Nov 03 21:59:15.036089 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:15.036043 6524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z" podUID="e0ef1acd-e793-4ea1-b89c-924d84be20bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.33:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Nov 03 21:59:15.196727 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:15.196680 6524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv" podUID="851596d1-02e4-42b9-bdab-19a9321a2a04" containerName="main" probeResult="failure" output="Get \"https://10.133.0.31:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Nov 03 21:59:15.197489 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:15.197465 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv" Nov 03 21:59:15.293815 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:15.293298 6524 patch_prober.go:29] interesting pod/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Nov 03 21:59:15.293815 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:15.293362 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" podUID="779feacc-19e1-4b9e-9968-896c178c78f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Nov 03 21:59:15.437297 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:15.437241 6524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j" podUID="7df4ae2c-a36f-4933-9e10-1f712fb47595" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Nov 03 21:59:15.656033 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:15.655992 6524 generic.go:356] "Generic (PLEG): container finished" podID="4e9aba93-9243-44df-94e7-33ca432c011b" containerID="17157d311048333f1f132b4f014d7f67cbe56fd3bc74a6d4752fd1ab32364717" exitCode=0 Nov 03 21:59:15.656370 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:15.656344 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-50e65b6f-k2hvk" event={"ID":"4e9aba93-9243-44df-94e7-33ca432c011b","Type":"ContainerDied","Data":"17157d311048333f1f132b4f014d7f67cbe56fd3bc74a6d4752fd1ab32364717"} Nov 03 21:59:15.738739 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:15.738692 6524 patch_prober.go:29] interesting pod/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Nov 03 21:59:15.738995 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:15.738962 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" podUID="efcc7e46-86ce-427c-a5d7-99878a682ac1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Nov 03 21:59:15.891654 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:15.891616 6524 patch_prober.go:29] interesting pod/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Nov 03 21:59:15.891930 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:15.891889 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" podUID="1d3e9770-476c-47a5-91d7-afffb476b8c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Nov 03 21:59:15.944372 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:15.944255 6524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf" podUID="1ab01063-2737-4b29-ade3-ef9cc00ab5e1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.32:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Nov 03 21:59:15.945027 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:15.945003 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf" Nov 03 21:59:16.039319 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:16.039265 6524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z" podUID="e0ef1acd-e793-4ea1-b89c-924d84be20bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.33:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Nov 03 21:59:16.050326 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:16.050283 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z" Nov 03 21:59:16.165207 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:16.165167 6524 patch_prober.go:29] interesting pod/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 03 21:59:16.165531 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:16.165476 6524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" podUID="8462643c-89e1-4ee3-a413-44160da8e03d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 03 21:59:16.224234 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:16.216242 6524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv" podUID="851596d1-02e4-42b9-bdab-19a9321a2a04" containerName="main" probeResult="failure" output="Get \"https://10.133.0.31:8000/ready\": context deadline exceeded" Nov 03 21:59:16.325147 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:16.286510 6524 patch_prober.go:29] interesting pod/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 03 21:59:16.325147 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:16.286583 6524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" podUID="7a83deb0-69e5-4c9d-a573-235be3ff17ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 03 21:59:16.325147 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:16.287207 6524 patch_prober.go:29] interesting pod/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Nov 03 21:59:16.325147 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:16.287250 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" podUID="779feacc-19e1-4b9e-9968-896c178c78f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Nov 03 21:59:16.445304 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:16.445260 6524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j" podUID="7df4ae2c-a36f-4933-9e10-1f712fb47595" containerName="main" probeResult="failure" output="Get \"https://10.133.0.34:8000/ready\": context deadline exceeded" Nov 03 21:59:16.446263 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:16.446237 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j" Nov 03 21:59:16.615145 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:16.610773 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z" Nov 03 21:59:16.628145 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:16.627512 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv" Nov 03 21:59:16.959332 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:16.959220 6524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf" podUID="1ab01063-2737-4b29-ade3-ef9cc00ab5e1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.32:8000/ready\": context deadline exceeded" Nov 03 21:59:17.067068 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:17.046228 6524 patch_prober.go:29] interesting pod/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 03 21:59:17.067068 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:17.046297 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" podUID="a9593fbc-9af2-4c1f-a986-3dc48dab11a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 03 21:59:17.235561 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:17.226597 6524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" Nov 03 21:59:17.235561 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:17.227936 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" Nov 03 21:59:17.235561 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:17.234615 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" Nov 03 21:59:17.236357 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:17.236191 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j" Nov 03 21:59:17.273346 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:17.272872 6524 patch_prober.go:29] interesting pod/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Nov 03 21:59:17.273346 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:17.272931 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" podUID="779feacc-19e1-4b9e-9968-896c178c78f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Nov 03 21:59:17.870150 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:17.845365 6524 patch_prober.go:29] interesting pod/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 03 21:59:17.870469 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:17.870428 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" podUID="efcc7e46-86ce-427c-a5d7-99878a682ac1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 03 21:59:17.999149 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:17.998330 6524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf" podUID="1ab01063-2737-4b29-ade3-ef9cc00ab5e1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.32:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Nov 03 21:59:18.053204 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:18.052202 6524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-2jqqp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.24:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 03 21:59:18.053204 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:18.052262 6524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-2jqqp" podUID="a85e1f40-b852-4024-a1eb-0c5798b081e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.24:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 03 21:59:18.101241 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:18.101199 6524 patch_prober.go:29] interesting pod/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 03 21:59:18.101493 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:18.101462 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" podUID="1d3e9770-476c-47a5-91d7-afffb476b8c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 03 21:59:18.224317 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:18.224259 6524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv" podUID="851596d1-02e4-42b9-bdab-19a9321a2a04" containerName="main" probeResult="failure" output="Get \"https://10.133.0.31:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Nov 03 21:59:18.357236 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:18.350720 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-50e65b6f-k2hvk" Nov 03 21:59:18.362228 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:18.359591 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf" Nov 03 21:59:18.510148 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:18.487646 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4e9aba93-9243-44df-94e7-33ca432c011b-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4e9aba93-9243-44df-94e7-33ca432c011b-maas-api-xks-ca\") pod \"4e9aba93-9243-44df-94e7-33ca432c011b\" (UID: \"4e9aba93-9243-44df-94e7-33ca432c011b\") " Nov 03 21:59:18.510148 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:18.487716 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4e9aba93-9243-44df-94e7-33ca432c011b-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4e9aba93-9243-44df-94e7-33ca432c011b-maas-api-openshift-ca\") pod \"4e9aba93-9243-44df-94e7-33ca432c011b\" (UID: \"4e9aba93-9243-44df-94e7-33ca432c011b\") " Nov 03 21:59:18.510148 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:18.487745 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4e9aba93-9243-44df-94e7-33ca432c011b-kube-api-access-bnvzw\" (UniqueName: \"kubernetes.io/projected/4e9aba93-9243-44df-94e7-33ca432c011b-kube-api-access-bnvzw\") pod \"4e9aba93-9243-44df-94e7-33ca432c011b\" (UID: \"4e9aba93-9243-44df-94e7-33ca432c011b\") " Nov 03 21:59:18.512615 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:18.512565 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e9aba93-9243-44df-94e7-33ca432c011b-maas-api-openshift-ca" pod "4e9aba93-9243-44df-94e7-33ca432c011b" (UID: "4e9aba93-9243-44df-94e7-33ca432c011b"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 03 21:59:18.661212 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:18.589850 6524 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4e9aba93-9243-44df-94e7-33ca432c011b-maas-api-openshift-ca\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:18.730552 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:18.730510 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e9aba93-9243-44df-94e7-33ca432c011b-kube-api-access-bnvzw" pod "4e9aba93-9243-44df-94e7-33ca432c011b" (UID: "4e9aba93-9243-44df-94e7-33ca432c011b"). InnerVolumeSpecName "kube-api-access-bnvzw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 03 21:59:18.754523 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:18.754489 6524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" Nov 03 21:59:18.838162 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:18.805299 6524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bnvzw\" (UniqueName: \"kubernetes.io/projected/4e9aba93-9243-44df-94e7-33ca432c011b-kube-api-access-bnvzw\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:18.838162 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:18.834669 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e9aba93-9243-44df-94e7-33ca432c011b-maas-api-xks-ca" pod "4e9aba93-9243-44df-94e7-33ca432c011b" (UID: "4e9aba93-9243-44df-94e7-33ca432c011b"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 03 21:59:18.853547 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:18.853514 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" Nov 03 21:59:18.853852 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:18.853832 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" Nov 03 21:59:18.909082 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:18.909034 6524 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4e9aba93-9243-44df-94e7-33ca432c011b-maas-api-xks-ca\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:18.916695 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:18.915215 6524 patch_prober.go:29] interesting pod/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 03 21:59:18.916695 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:18.915272 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" podUID="efcc7e46-86ce-427c-a5d7-99878a682ac1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 03 21:59:19.106105 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:19.106064 6524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-f4fb1b-gxdmt"] Nov 03 21:59:19.144218 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:19.144181 6524 cpu_manager.go:414] "containerMap: removing container" podUID="4e9aba93-9243-44df-94e7-33ca432c011b" containerName="revoke-keys" Nov 03 21:59:19.144469 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:19.144447 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4e9aba93-9243-44df-94e7-33ca432c011b" containerName="revoke-keys" Nov 03 21:59:19.148298 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:19.148267 6524 patch_prober.go:29] interesting pod/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 03 21:59:19.148517 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:19.148473 6524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" podUID="1d3e9770-476c-47a5-91d7-afffb476b8c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 03 21:59:19.194586 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:19.194493 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="4e9aba93-9243-44df-94e7-33ca432c011b" containerName="revoke-keys" Nov 03 21:59:19.237777 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:19.237739 6524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" Nov 03 21:59:19.247177 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:19.238499 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-f4fb1b-gxdmt" Nov 03 21:59:19.247177 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:19.239095 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" Nov 03 21:59:19.254891 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:19.254220 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-f4fb1b-gxdmt"] Nov 03 21:59:19.304763 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:19.303919 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-50e65b6f-k2hvk" Nov 03 21:59:19.304763 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:19.304410 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-50e65b6f-k2hvk" event={"ID":"4e9aba93-9243-44df-94e7-33ca432c011b","Type":"ContainerDied","Data":"94fe36687786f7a836b046b4336eb96838100c47678b7985e5bfa06c41d602fb"} Nov 03 21:59:19.304763 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:19.304455 6524 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="94fe36687786f7a836b046b4336eb96838100c47678b7985e5bfa06c41d602fb" Nov 03 21:59:19.341155 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:19.334822 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7a080818-a714-41a6-b2e1-b433512dc395-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-f4fb1b-gxdmt\" (UID: \"7a080818-a714-41a6-b2e1-b433512dc395\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-f4fb1b-gxdmt" Nov 03 21:59:19.341663 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:19.341629 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7a080818-a714-41a6-b2e1-b433512dc395-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-f4fb1b-gxdmt\" (UID: \"7a080818-a714-41a6-b2e1-b433512dc395\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-f4fb1b-gxdmt" Nov 03 21:59:19.353479 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:19.353434 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dz67g\" (UniqueName: \"kubernetes.io/projected/7a080818-a714-41a6-b2e1-b433512dc395-kube-api-access-dz67g\") pod \"maas-api-revoke-keys-e2e-worker-w1-f4fb1b-gxdmt\" (UID: \"7a080818-a714-41a6-b2e1-b433512dc395\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-f4fb1b-gxdmt" Nov 03 21:59:19.391583 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:19.391542 6524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" Nov 03 21:59:19.393826 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:19.393803 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" Nov 03 21:59:19.456621 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:19.456525 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7a080818-a714-41a6-b2e1-b433512dc395-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-f4fb1b-gxdmt\" (UID: \"7a080818-a714-41a6-b2e1-b433512dc395\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-f4fb1b-gxdmt" Nov 03 21:59:19.474376 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:19.474339 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dz67g\" (UniqueName: \"kubernetes.io/projected/7a080818-a714-41a6-b2e1-b433512dc395-kube-api-access-dz67g\") pod \"maas-api-revoke-keys-e2e-worker-w1-f4fb1b-gxdmt\" (UID: \"7a080818-a714-41a6-b2e1-b433512dc395\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-f4fb1b-gxdmt" Nov 03 21:59:19.475758 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:19.475731 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7a080818-a714-41a6-b2e1-b433512dc395-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-f4fb1b-gxdmt\" (UID: \"7a080818-a714-41a6-b2e1-b433512dc395\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-f4fb1b-gxdmt" Nov 03 21:59:19.476944 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:19.476914 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7a080818-a714-41a6-b2e1-b433512dc395-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-f4fb1b-gxdmt\" (UID: \"7a080818-a714-41a6-b2e1-b433512dc395\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-f4fb1b-gxdmt" Nov 03 21:59:19.484652 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:19.484609 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf"] Nov 03 21:59:19.490342 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:19.490311 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7a080818-a714-41a6-b2e1-b433512dc395-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-f4fb1b-gxdmt\" (UID: \"7a080818-a714-41a6-b2e1-b433512dc395\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-f4fb1b-gxdmt" Nov 03 21:59:19.497356 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:19.497313 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dz67g\" (UniqueName: \"kubernetes.io/projected/7a080818-a714-41a6-b2e1-b433512dc395-kube-api-access-dz67g\") pod \"maas-api-revoke-keys-e2e-worker-w1-f4fb1b-gxdmt\" (UID: \"7a080818-a714-41a6-b2e1-b433512dc395\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-f4fb1b-gxdmt" Nov 03 21:59:19.552736 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:19.552689 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" Nov 03 21:59:19.581179 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:19.580272 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" Nov 03 21:59:19.640223 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:19.639864 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-f4fb1b-gxdmt" Nov 03 21:59:19.938059 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:19.937674 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-f4fb1b-gxdmt"] Nov 03 21:59:19.940435 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:59:19.940403 6524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a080818_a714_41a6_b2e1_b433512dc395.slice/crio-3e2c1bd7903f8f736071fd5faee68b63076cfb2be654f4564ff18b9c43da965f WatchSource:0}: Error finding container 3e2c1bd7903f8f736071fd5faee68b63076cfb2be654f4564ff18b9c43da965f: Status 404 returned error can't find the container with id 3e2c1bd7903f8f736071fd5faee68b63076cfb2be654f4564ff18b9c43da965f Nov 03 21:59:20.310919 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:20.310828 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-f4fb1b-gxdmt" event={"ID":"7a080818-a714-41a6-b2e1-b433512dc395","Type":"ContainerStarted","Data":"8c0edac6b21a3e13967c8bd34431b0bf053b30adcf156f8d45eb7f976f84935f"} Nov 03 21:59:20.310919 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:20.310884 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-f4fb1b-gxdmt" event={"ID":"7a080818-a714-41a6-b2e1-b433512dc395","Type":"ContainerStarted","Data":"3e2c1bd7903f8f736071fd5faee68b63076cfb2be654f4564ff18b9c43da965f"} Nov 03 21:59:20.343150 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:20.339575 6524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-f4fb1b-gxdmt" podStartSLOduration=2.339550965 podStartE2EDuration="2.339550965s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:59:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:59:20.338586433 +0000 UTC m=+805.690913239" watchObservedRunningTime="2026-11-03 21:59:20.339550965 +0000 UTC m=+805.691877769" Nov 03 21:59:21.317784 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:21.317746 6524 generic.go:356] "Generic (PLEG): container finished" podID="7a080818-a714-41a6-b2e1-b433512dc395" containerID="8c0edac6b21a3e13967c8bd34431b0bf053b30adcf156f8d45eb7f976f84935f" exitCode=0 Nov 03 21:59:21.318239 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:21.317807 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-f4fb1b-gxdmt" event={"ID":"7a080818-a714-41a6-b2e1-b433512dc395","Type":"ContainerDied","Data":"8c0edac6b21a3e13967c8bd34431b0bf053b30adcf156f8d45eb7f976f84935f"} Nov 03 21:59:22.456683 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:22.456652 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-f4fb1b-gxdmt" Nov 03 21:59:22.519862 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:22.519823 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7a080818-a714-41a6-b2e1-b433512dc395-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7a080818-a714-41a6-b2e1-b433512dc395-maas-api-xks-ca\") pod \"7a080818-a714-41a6-b2e1-b433512dc395\" (UID: \"7a080818-a714-41a6-b2e1-b433512dc395\") " Nov 03 21:59:22.520035 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:22.519908 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7a080818-a714-41a6-b2e1-b433512dc395-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7a080818-a714-41a6-b2e1-b433512dc395-maas-api-openshift-ca\") pod \"7a080818-a714-41a6-b2e1-b433512dc395\" (UID: \"7a080818-a714-41a6-b2e1-b433512dc395\") " Nov 03 21:59:22.520035 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:22.520007 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7a080818-a714-41a6-b2e1-b433512dc395-kube-api-access-dz67g\" (UniqueName: \"kubernetes.io/projected/7a080818-a714-41a6-b2e1-b433512dc395-kube-api-access-dz67g\") pod \"7a080818-a714-41a6-b2e1-b433512dc395\" (UID: \"7a080818-a714-41a6-b2e1-b433512dc395\") " Nov 03 21:59:22.520357 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:22.520326 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a080818-a714-41a6-b2e1-b433512dc395-maas-api-openshift-ca" pod "7a080818-a714-41a6-b2e1-b433512dc395" (UID: "7a080818-a714-41a6-b2e1-b433512dc395"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 03 21:59:22.521963 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:22.521936 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a080818-a714-41a6-b2e1-b433512dc395-maas-api-xks-ca" pod "7a080818-a714-41a6-b2e1-b433512dc395" (UID: "7a080818-a714-41a6-b2e1-b433512dc395"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 03 21:59:22.522080 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:22.522062 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a080818-a714-41a6-b2e1-b433512dc395-kube-api-access-dz67g" pod "7a080818-a714-41a6-b2e1-b433512dc395" (UID: "7a080818-a714-41a6-b2e1-b433512dc395"). InnerVolumeSpecName "kube-api-access-dz67g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 03 21:59:22.621996 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:22.621959 6524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dz67g\" (UniqueName: \"kubernetes.io/projected/7a080818-a714-41a6-b2e1-b433512dc395-kube-api-access-dz67g\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:22.621996 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:22.621997 6524 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7a080818-a714-41a6-b2e1-b433512dc395-maas-api-xks-ca\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:22.622236 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:22.622013 6524 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7a080818-a714-41a6-b2e1-b433512dc395-maas-api-openshift-ca\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:23.323031 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:23.322987 6524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-02e8ab-865jn"] Nov 03 21:59:23.323447 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:23.323430 6524 cpu_manager.go:414] "containerMap: removing container" podUID="7a080818-a714-41a6-b2e1-b433512dc395" containerName="revoke-keys" Nov 03 21:59:23.323447 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:23.323446 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7a080818-a714-41a6-b2e1-b433512dc395" containerName="revoke-keys" Nov 03 21:59:23.323596 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:23.323519 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="7a080818-a714-41a6-b2e1-b433512dc395" containerName="revoke-keys" Nov 03 21:59:23.326933 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:23.326909 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-02e8ab-865jn" Nov 03 21:59:23.331426 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:23.331403 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-f4fb1b-gxdmt" Nov 03 21:59:23.331575 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:23.331398 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-f4fb1b-gxdmt" event={"ID":"7a080818-a714-41a6-b2e1-b433512dc395","Type":"ContainerDied","Data":"3e2c1bd7903f8f736071fd5faee68b63076cfb2be654f4564ff18b9c43da965f"} Nov 03 21:59:23.331575 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:23.331517 6524 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3e2c1bd7903f8f736071fd5faee68b63076cfb2be654f4564ff18b9c43da965f" Nov 03 21:59:23.335522 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:23.335494 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-02e8ab-865jn"] Nov 03 21:59:23.430397 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:23.430357 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4m42\" (UniqueName: \"kubernetes.io/projected/3a50052d-56d1-451b-8762-1f81880a4406-kube-api-access-f4m42\") pod \"maas-api-revoke-keys-e2e-worker-w3-02e8ab-865jn\" (UID: \"3a50052d-56d1-451b-8762-1f81880a4406\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-02e8ab-865jn" Nov 03 21:59:23.430589 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:23.430423 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3a50052d-56d1-451b-8762-1f81880a4406-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-02e8ab-865jn\" (UID: \"3a50052d-56d1-451b-8762-1f81880a4406\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-02e8ab-865jn" Nov 03 21:59:23.430589 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:23.430500 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3a50052d-56d1-451b-8762-1f81880a4406-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-02e8ab-865jn\" (UID: \"3a50052d-56d1-451b-8762-1f81880a4406\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-02e8ab-865jn" Nov 03 21:59:23.531987 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:23.531953 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4m42\" (UniqueName: \"kubernetes.io/projected/3a50052d-56d1-451b-8762-1f81880a4406-kube-api-access-f4m42\") pod \"maas-api-revoke-keys-e2e-worker-w3-02e8ab-865jn\" (UID: \"3a50052d-56d1-451b-8762-1f81880a4406\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-02e8ab-865jn" Nov 03 21:59:23.532373 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:23.532014 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3a50052d-56d1-451b-8762-1f81880a4406-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-02e8ab-865jn\" (UID: \"3a50052d-56d1-451b-8762-1f81880a4406\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-02e8ab-865jn" Nov 03 21:59:23.532373 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:23.532043 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3a50052d-56d1-451b-8762-1f81880a4406-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-02e8ab-865jn\" (UID: \"3a50052d-56d1-451b-8762-1f81880a4406\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-02e8ab-865jn" Nov 03 21:59:23.532703 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:23.532679 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3a50052d-56d1-451b-8762-1f81880a4406-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-02e8ab-865jn\" (UID: \"3a50052d-56d1-451b-8762-1f81880a4406\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-02e8ab-865jn" Nov 03 21:59:23.534547 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:23.534525 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3a50052d-56d1-451b-8762-1f81880a4406-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-02e8ab-865jn\" (UID: \"3a50052d-56d1-451b-8762-1f81880a4406\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-02e8ab-865jn" Nov 03 21:59:23.545089 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:23.545064 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4m42\" (UniqueName: \"kubernetes.io/projected/3a50052d-56d1-451b-8762-1f81880a4406-kube-api-access-f4m42\") pod \"maas-api-revoke-keys-e2e-worker-w3-02e8ab-865jn\" (UID: \"3a50052d-56d1-451b-8762-1f81880a4406\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-02e8ab-865jn" Nov 03 21:59:23.645037 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:23.644760 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-02e8ab-865jn" Nov 03 21:59:23.756371 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:23.756336 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j"] Nov 03 21:59:23.951609 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:23.951503 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-02e8ab-865jn"] Nov 03 21:59:23.953928 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:59:23.953902 6524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a50052d_56d1_451b_8762_1f81880a4406.slice/crio-7300ebd0c788066321a3ad3d95556a343630918c6d7511fced7d52d3b750caf7 WatchSource:0}: Error finding container 7300ebd0c788066321a3ad3d95556a343630918c6d7511fced7d52d3b750caf7: Status 404 returned error can't find the container with id 7300ebd0c788066321a3ad3d95556a343630918c6d7511fced7d52d3b750caf7 Nov 03 21:59:24.338424 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:24.338387 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-02e8ab-865jn" event={"ID":"3a50052d-56d1-451b-8762-1f81880a4406","Type":"ContainerStarted","Data":"9d2e643605388a091623a3934da3cfb02f64d9dcee2ca89210ab6ee2b23e4edf"} Nov 03 21:59:24.338596 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:24.338445 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-02e8ab-865jn" event={"ID":"3a50052d-56d1-451b-8762-1f81880a4406","Type":"ContainerStarted","Data":"7300ebd0c788066321a3ad3d95556a343630918c6d7511fced7d52d3b750caf7"} Nov 03 21:59:24.358206 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:24.358153 6524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-02e8ab-865jn" podStartSLOduration=1.358137106 podStartE2EDuration="1.358137106s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:59:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:59:24.356887422 +0000 UTC m=+809.709214217" watchObservedRunningTime="2026-11-03 21:59:24.358137106 +0000 UTC m=+809.710463907" Nov 03 21:59:25.344693 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:25.344649 6524 generic.go:356] "Generic (PLEG): container finished" podID="3a50052d-56d1-451b-8762-1f81880a4406" containerID="9d2e643605388a091623a3934da3cfb02f64d9dcee2ca89210ab6ee2b23e4edf" exitCode=0 Nov 03 21:59:25.344693 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:25.344761 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-02e8ab-865jn" event={"ID":"3a50052d-56d1-451b-8762-1f81880a4406","Type":"ContainerDied","Data":"9d2e643605388a091623a3934da3cfb02f64d9dcee2ca89210ab6ee2b23e4edf"} Nov 03 21:59:26.520893 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:26.520862 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-02e8ab-865jn" Nov 03 21:59:26.590664 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:26.590621 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3a50052d-56d1-451b-8762-1f81880a4406-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3a50052d-56d1-451b-8762-1f81880a4406-maas-api-xks-ca\") pod \"3a50052d-56d1-451b-8762-1f81880a4406\" (UID: \"3a50052d-56d1-451b-8762-1f81880a4406\") " Nov 03 21:59:26.590876 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:26.590730 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3a50052d-56d1-451b-8762-1f81880a4406-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3a50052d-56d1-451b-8762-1f81880a4406-maas-api-openshift-ca\") pod \"3a50052d-56d1-451b-8762-1f81880a4406\" (UID: \"3a50052d-56d1-451b-8762-1f81880a4406\") " Nov 03 21:59:26.590876 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:26.590765 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3a50052d-56d1-451b-8762-1f81880a4406-kube-api-access-f4m42\" (UniqueName: \"kubernetes.io/projected/3a50052d-56d1-451b-8762-1f81880a4406-kube-api-access-f4m42\") pod \"3a50052d-56d1-451b-8762-1f81880a4406\" (UID: \"3a50052d-56d1-451b-8762-1f81880a4406\") " Nov 03 21:59:26.591170 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:26.591143 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a50052d-56d1-451b-8762-1f81880a4406-maas-api-openshift-ca" pod "3a50052d-56d1-451b-8762-1f81880a4406" (UID: "3a50052d-56d1-451b-8762-1f81880a4406"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 03 21:59:26.592911 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:26.592876 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a50052d-56d1-451b-8762-1f81880a4406-maas-api-xks-ca" pod "3a50052d-56d1-451b-8762-1f81880a4406" (UID: "3a50052d-56d1-451b-8762-1f81880a4406"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 03 21:59:26.594132 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:26.594101 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a50052d-56d1-451b-8762-1f81880a4406-kube-api-access-f4m42" pod "3a50052d-56d1-451b-8762-1f81880a4406" (UID: "3a50052d-56d1-451b-8762-1f81880a4406"). InnerVolumeSpecName "kube-api-access-f4m42". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 03 21:59:26.692540 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:26.692450 6524 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3a50052d-56d1-451b-8762-1f81880a4406-maas-api-openshift-ca\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:26.692540 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:26.692483 6524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4m42\" (UniqueName: \"kubernetes.io/projected/3a50052d-56d1-451b-8762-1f81880a4406-kube-api-access-f4m42\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:26.692540 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:26.692496 6524 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3a50052d-56d1-451b-8762-1f81880a4406-maas-api-xks-ca\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:27.370606 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:27.370575 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-02e8ab-865jn" Nov 03 21:59:27.370795 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:27.370608 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-02e8ab-865jn" event={"ID":"3a50052d-56d1-451b-8762-1f81880a4406","Type":"ContainerDied","Data":"7300ebd0c788066321a3ad3d95556a343630918c6d7511fced7d52d3b750caf7"} Nov 03 21:59:27.370795 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:27.370666 6524 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7300ebd0c788066321a3ad3d95556a343630918c6d7511fced7d52d3b750caf7" Nov 03 21:59:28.167856 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:28.167810 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z"] Nov 03 21:59:28.880169 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:28.872408 6524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1dcacc9b-57xf9"] Nov 03 21:59:28.880169 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:28.873953 6524 cpu_manager.go:414] "containerMap: removing container" podUID="3a50052d-56d1-451b-8762-1f81880a4406" containerName="revoke-keys" Nov 03 21:59:28.880169 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:28.873978 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3a50052d-56d1-451b-8762-1f81880a4406" containerName="revoke-keys" Nov 03 21:59:28.880169 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:28.874660 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="3a50052d-56d1-451b-8762-1f81880a4406" containerName="revoke-keys" Nov 03 21:59:28.898781 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:28.895407 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1dcacc9b-57xf9"] Nov 03 21:59:28.898781 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:28.895562 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1dcacc9b-57xf9" Nov 03 21:59:28.931725 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:28.931699 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Nov 03 21:59:28.937288 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:28.932374 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5n7j7\"" Nov 03 21:59:29.084300 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:29.084254 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mt2tt\" (UniqueName: \"kubernetes.io/projected/86b62a38-13e6-432c-b288-4668c256205e-kube-api-access-mt2tt\") pod \"maas-api-revoke-sub-1dcacc9b-57xf9\" (UID: \"86b62a38-13e6-432c-b288-4668c256205e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1dcacc9b-57xf9" Nov 03 21:59:29.084462 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:29.084443 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/86b62a38-13e6-432c-b288-4668c256205e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1dcacc9b-57xf9\" (UID: \"86b62a38-13e6-432c-b288-4668c256205e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1dcacc9b-57xf9" Nov 03 21:59:29.084619 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:29.084590 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/86b62a38-13e6-432c-b288-4668c256205e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1dcacc9b-57xf9\" (UID: \"86b62a38-13e6-432c-b288-4668c256205e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1dcacc9b-57xf9" Nov 03 21:59:29.185522 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:29.185418 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mt2tt\" (UniqueName: \"kubernetes.io/projected/86b62a38-13e6-432c-b288-4668c256205e-kube-api-access-mt2tt\") pod \"maas-api-revoke-sub-1dcacc9b-57xf9\" (UID: \"86b62a38-13e6-432c-b288-4668c256205e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1dcacc9b-57xf9" Nov 03 21:59:29.185522 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:29.185512 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/86b62a38-13e6-432c-b288-4668c256205e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1dcacc9b-57xf9\" (UID: \"86b62a38-13e6-432c-b288-4668c256205e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1dcacc9b-57xf9" Nov 03 21:59:29.186070 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:29.185564 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/86b62a38-13e6-432c-b288-4668c256205e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1dcacc9b-57xf9\" (UID: \"86b62a38-13e6-432c-b288-4668c256205e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1dcacc9b-57xf9" Nov 03 21:59:29.186249 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:29.186222 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/86b62a38-13e6-432c-b288-4668c256205e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1dcacc9b-57xf9\" (UID: \"86b62a38-13e6-432c-b288-4668c256205e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1dcacc9b-57xf9" Nov 03 21:59:29.190408 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:29.190311 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/86b62a38-13e6-432c-b288-4668c256205e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1dcacc9b-57xf9\" (UID: \"86b62a38-13e6-432c-b288-4668c256205e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1dcacc9b-57xf9" Nov 03 21:59:29.203883 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:29.203835 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mt2tt\" (UniqueName: \"kubernetes.io/projected/86b62a38-13e6-432c-b288-4668c256205e-kube-api-access-mt2tt\") pod \"maas-api-revoke-sub-1dcacc9b-57xf9\" (UID: \"86b62a38-13e6-432c-b288-4668c256205e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1dcacc9b-57xf9" Nov 03 21:59:29.267160 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:29.261825 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1dcacc9b-57xf9" Nov 03 21:59:29.694149 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:29.679749 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1dcacc9b-57xf9"] Nov 03 21:59:29.741170 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:59:29.735921 6524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86b62a38_13e6_432c_b288_4668c256205e.slice/crio-41bfa545960b63054d4be72b080ae0f51131af6f934bfababb4325d99fb00a74 WatchSource:0}: Error finding container 41bfa545960b63054d4be72b080ae0f51131af6f934bfababb4325d99fb00a74: Status 404 returned error can't find the container with id 41bfa545960b63054d4be72b080ae0f51131af6f934bfababb4325d99fb00a74 Nov 03 21:59:29.867087 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:59:29.863549 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 21:59:30.426338 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:30.425305 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1dcacc9b-57xf9" event={"ID":"86b62a38-13e6-432c-b288-4668c256205e","Type":"ContainerStarted","Data":"1c4c0864c88a507bba31db68196125ed7a9f3476b39e12b3df7f686ccb13818c"} Nov 03 21:59:30.426338 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:30.425380 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1dcacc9b-57xf9" event={"ID":"86b62a38-13e6-432c-b288-4668c256205e","Type":"ContainerStarted","Data":"41bfa545960b63054d4be72b080ae0f51131af6f934bfababb4325d99fb00a74"} Nov 03 21:59:30.463288 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:30.461386 6524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1dcacc9b-57xf9" podStartSLOduration=2.461366485 podStartE2EDuration="2.461366485s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:59:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:59:30.458557719 +0000 UTC m=+815.810884549" watchObservedRunningTime="2026-11-03 21:59:30.461366485 +0000 UTC m=+815.813693291" Nov 03 21:59:31.079195 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:31.078647 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-8af3f3-655f44685b-wh9n2"] Nov 03 21:59:31.081819 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:31.081533 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-8af3f3-7c9fcbb455-25425"] Nov 03 21:59:31.445148 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:31.441348 6524 generic.go:356] "Generic (PLEG): container finished" podID="86b62a38-13e6-432c-b288-4668c256205e" containerID="1c4c0864c88a507bba31db68196125ed7a9f3476b39e12b3df7f686ccb13818c" exitCode=0 Nov 03 21:59:31.445148 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:31.441417 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1dcacc9b-57xf9" event={"ID":"86b62a38-13e6-432c-b288-4668c256205e","Type":"ContainerDied","Data":"1c4c0864c88a507bba31db68196125ed7a9f3476b39e12b3df7f686ccb13818c"} Nov 03 21:59:32.667208 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:32.667177 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1dcacc9b-57xf9" Nov 03 21:59:32.803893 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:32.803800 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/86b62a38-13e6-432c-b288-4668c256205e-kube-api-access-mt2tt\" (UniqueName: \"kubernetes.io/projected/86b62a38-13e6-432c-b288-4668c256205e-kube-api-access-mt2tt\") pod \"86b62a38-13e6-432c-b288-4668c256205e\" (UID: \"86b62a38-13e6-432c-b288-4668c256205e\") " Nov 03 21:59:32.804055 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:32.803945 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/86b62a38-13e6-432c-b288-4668c256205e-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/86b62a38-13e6-432c-b288-4668c256205e-maas-api-openshift-ca\") pod \"86b62a38-13e6-432c-b288-4668c256205e\" (UID: \"86b62a38-13e6-432c-b288-4668c256205e\") " Nov 03 21:59:32.804055 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:32.804002 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/86b62a38-13e6-432c-b288-4668c256205e-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/86b62a38-13e6-432c-b288-4668c256205e-maas-api-xks-ca\") pod \"86b62a38-13e6-432c-b288-4668c256205e\" (UID: \"86b62a38-13e6-432c-b288-4668c256205e\") " Nov 03 21:59:32.804360 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:32.804331 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86b62a38-13e6-432c-b288-4668c256205e-maas-api-openshift-ca" pod "86b62a38-13e6-432c-b288-4668c256205e" (UID: "86b62a38-13e6-432c-b288-4668c256205e"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 03 21:59:32.805887 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:32.805863 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86b62a38-13e6-432c-b288-4668c256205e-kube-api-access-mt2tt" pod "86b62a38-13e6-432c-b288-4668c256205e" (UID: "86b62a38-13e6-432c-b288-4668c256205e"). InnerVolumeSpecName "kube-api-access-mt2tt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 03 21:59:32.805951 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:32.805930 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86b62a38-13e6-432c-b288-4668c256205e-maas-api-xks-ca" pod "86b62a38-13e6-432c-b288-4668c256205e" (UID: "86b62a38-13e6-432c-b288-4668c256205e"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 03 21:59:32.904838 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:32.904794 6524 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/86b62a38-13e6-432c-b288-4668c256205e-maas-api-xks-ca\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:32.904838 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:32.904833 6524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mt2tt\" (UniqueName: \"kubernetes.io/projected/86b62a38-13e6-432c-b288-4668c256205e-kube-api-access-mt2tt\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:32.904838 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:32.904845 6524 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/86b62a38-13e6-432c-b288-4668c256205e-maas-api-openshift-ca\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:33.503064 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:33.503025 6524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e85a4dd4-zn26z"] Nov 03 21:59:33.503477 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:33.503460 6524 cpu_manager.go:414] "containerMap: removing container" podUID="86b62a38-13e6-432c-b288-4668c256205e" containerName="revoke-keys" Nov 03 21:59:33.503477 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:33.503475 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="86b62a38-13e6-432c-b288-4668c256205e" containerName="revoke-keys" Nov 03 21:59:33.503641 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:33.503572 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="86b62a38-13e6-432c-b288-4668c256205e" containerName="revoke-keys" Nov 03 21:59:33.507055 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:33.507019 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e85a4dd4-zn26z" Nov 03 21:59:33.521152 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:33.521087 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1dcacc9b-57xf9" event={"ID":"86b62a38-13e6-432c-b288-4668c256205e","Type":"ContainerDied","Data":"41bfa545960b63054d4be72b080ae0f51131af6f934bfababb4325d99fb00a74"} Nov 03 21:59:33.521295 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:33.521175 6524 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="41bfa545960b63054d4be72b080ae0f51131af6f934bfababb4325d99fb00a74" Nov 03 21:59:33.521295 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:33.521257 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1dcacc9b-57xf9" Nov 03 21:59:33.521375 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:33.521311 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e85a4dd4-zn26z"] Nov 03 21:59:33.612295 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:33.612261 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2862v\" (UniqueName: \"kubernetes.io/projected/fdad2bdf-ab1d-4800-9557-3e9b88953a80-kube-api-access-2862v\") pod \"maas-api-revoke-sub-e85a4dd4-zn26z\" (UID: \"fdad2bdf-ab1d-4800-9557-3e9b88953a80\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e85a4dd4-zn26z" Nov 03 21:59:33.612478 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:33.612318 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fdad2bdf-ab1d-4800-9557-3e9b88953a80-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e85a4dd4-zn26z\" (UID: \"fdad2bdf-ab1d-4800-9557-3e9b88953a80\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e85a4dd4-zn26z" Nov 03 21:59:33.612527 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:33.612487 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fdad2bdf-ab1d-4800-9557-3e9b88953a80-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e85a4dd4-zn26z\" (UID: \"fdad2bdf-ab1d-4800-9557-3e9b88953a80\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e85a4dd4-zn26z" Nov 03 21:59:33.714163 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:33.714101 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fdad2bdf-ab1d-4800-9557-3e9b88953a80-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e85a4dd4-zn26z\" (UID: \"fdad2bdf-ab1d-4800-9557-3e9b88953a80\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e85a4dd4-zn26z" Nov 03 21:59:33.714598 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:33.714247 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2862v\" (UniqueName: \"kubernetes.io/projected/fdad2bdf-ab1d-4800-9557-3e9b88953a80-kube-api-access-2862v\") pod \"maas-api-revoke-sub-e85a4dd4-zn26z\" (UID: \"fdad2bdf-ab1d-4800-9557-3e9b88953a80\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e85a4dd4-zn26z" Nov 03 21:59:33.714598 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:33.714287 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fdad2bdf-ab1d-4800-9557-3e9b88953a80-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e85a4dd4-zn26z\" (UID: \"fdad2bdf-ab1d-4800-9557-3e9b88953a80\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e85a4dd4-zn26z" Nov 03 21:59:33.714770 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:33.714748 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fdad2bdf-ab1d-4800-9557-3e9b88953a80-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e85a4dd4-zn26z\" (UID: \"fdad2bdf-ab1d-4800-9557-3e9b88953a80\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e85a4dd4-zn26z" Nov 03 21:59:33.716659 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:33.716635 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fdad2bdf-ab1d-4800-9557-3e9b88953a80-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e85a4dd4-zn26z\" (UID: \"fdad2bdf-ab1d-4800-9557-3e9b88953a80\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e85a4dd4-zn26z" Nov 03 21:59:33.729839 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:33.729809 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2862v\" (UniqueName: \"kubernetes.io/projected/fdad2bdf-ab1d-4800-9557-3e9b88953a80-kube-api-access-2862v\") pod \"maas-api-revoke-sub-e85a4dd4-zn26z\" (UID: \"fdad2bdf-ab1d-4800-9557-3e9b88953a80\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e85a4dd4-zn26z" Nov 03 21:59:33.832279 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:33.828441 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e85a4dd4-zn26z" Nov 03 21:59:34.158018 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:34.157989 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e85a4dd4-zn26z"] Nov 03 21:59:34.160072 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:59:34.160046 6524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdad2bdf_ab1d_4800_9557_3e9b88953a80.slice/crio-666813f28d51773c11a25ccf93b7692e38461cb20a8fb009d1de0a8b6b9d7696 WatchSource:0}: Error finding container 666813f28d51773c11a25ccf93b7692e38461cb20a8fb009d1de0a8b6b9d7696: Status 404 returned error can't find the container with id 666813f28d51773c11a25ccf93b7692e38461cb20a8fb009d1de0a8b6b9d7696 Nov 03 21:59:34.530635 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:34.530588 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e85a4dd4-zn26z" event={"ID":"fdad2bdf-ab1d-4800-9557-3e9b88953a80","Type":"ContainerStarted","Data":"7945b79fc07623b834f9549b92ffb9077c6db865b73179e2fcdf31c8723fa1a2"} Nov 03 21:59:34.530779 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:34.530657 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e85a4dd4-zn26z" event={"ID":"fdad2bdf-ab1d-4800-9557-3e9b88953a80","Type":"ContainerStarted","Data":"666813f28d51773c11a25ccf93b7692e38461cb20a8fb009d1de0a8b6b9d7696"} Nov 03 21:59:34.546471 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:34.546442 6524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf" podUID="1ab01063-2737-4b29-ade3-ef9cc00ab5e1" containerName="main" containerID="cri-o://fe389a2e12e1ea5918c87e8a762d1c1f8f166b701ce9d3b273797bd9e6d30358" gracePeriod=45 Nov 03 21:59:34.552847 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:34.552801 6524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e85a4dd4-zn26z" podStartSLOduration=1.5527873140000001 podStartE2EDuration="1.552787314s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:59:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:59:34.54961546 +0000 UTC m=+819.901942264" watchObservedRunningTime="2026-11-03 21:59:34.552787314 +0000 UTC m=+819.905114118" Nov 03 21:59:34.836632 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:34.836604 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf" Nov 03 21:59:34.939999 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:34.939962 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ab01063-2737-4b29-ade3-ef9cc00ab5e1-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ab01063-2737-4b29-ade3-ef9cc00ab5e1-tmp-dir\") pod \"1ab01063-2737-4b29-ade3-ef9cc00ab5e1\" (UID: \"1ab01063-2737-4b29-ade3-ef9cc00ab5e1\") " Nov 03 21:59:34.940196 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:34.940012 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ab01063-2737-4b29-ade3-ef9cc00ab5e1-home\" (UniqueName: \"kubernetes.io/empty-dir/1ab01063-2737-4b29-ade3-ef9cc00ab5e1-home\") pod \"1ab01063-2737-4b29-ade3-ef9cc00ab5e1\" (UID: \"1ab01063-2737-4b29-ade3-ef9cc00ab5e1\") " Nov 03 21:59:34.940196 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:34.940035 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1ab01063-2737-4b29-ade3-ef9cc00ab5e1-tls-certs\" (UniqueName: \"kubernetes.io/secret/1ab01063-2737-4b29-ade3-ef9cc00ab5e1-tls-certs\") pod \"1ab01063-2737-4b29-ade3-ef9cc00ab5e1\" (UID: \"1ab01063-2737-4b29-ade3-ef9cc00ab5e1\") " Nov 03 21:59:34.940196 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:34.940093 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ab01063-2737-4b29-ade3-ef9cc00ab5e1-dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ab01063-2737-4b29-ade3-ef9cc00ab5e1-dshm\") pod \"1ab01063-2737-4b29-ade3-ef9cc00ab5e1\" (UID: \"1ab01063-2737-4b29-ade3-ef9cc00ab5e1\") " Nov 03 21:59:34.940377 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:34.940225 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ab01063-2737-4b29-ade3-ef9cc00ab5e1-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ab01063-2737-4b29-ade3-ef9cc00ab5e1-model-cache\") pod \"1ab01063-2737-4b29-ade3-ef9cc00ab5e1\" (UID: \"1ab01063-2737-4b29-ade3-ef9cc00ab5e1\") " Nov 03 21:59:34.940377 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:34.940300 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1ab01063-2737-4b29-ade3-ef9cc00ab5e1-kube-api-access-cj2kh\" (UniqueName: \"kubernetes.io/projected/1ab01063-2737-4b29-ade3-ef9cc00ab5e1-kube-api-access-cj2kh\") pod \"1ab01063-2737-4b29-ade3-ef9cc00ab5e1\" (UID: \"1ab01063-2737-4b29-ade3-ef9cc00ab5e1\") " Nov 03 21:59:34.940377 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:34.940314 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ab01063-2737-4b29-ade3-ef9cc00ab5e1-tmp-dir" pod "1ab01063-2737-4b29-ade3-ef9cc00ab5e1" (UID: "1ab01063-2737-4b29-ade3-ef9cc00ab5e1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 21:59:34.940377 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:34.940353 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ab01063-2737-4b29-ade3-ef9cc00ab5e1-home" pod "1ab01063-2737-4b29-ade3-ef9cc00ab5e1" (UID: "1ab01063-2737-4b29-ade3-ef9cc00ab5e1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 21:59:34.940577 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:34.940500 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ab01063-2737-4b29-ade3-ef9cc00ab5e1-model-cache" pod "1ab01063-2737-4b29-ade3-ef9cc00ab5e1" (UID: "1ab01063-2737-4b29-ade3-ef9cc00ab5e1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 21:59:34.940738 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:34.940722 6524 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ab01063-2737-4b29-ade3-ef9cc00ab5e1-model-cache\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:34.940795 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:34.940742 6524 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ab01063-2737-4b29-ade3-ef9cc00ab5e1-tmp-dir\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:34.940795 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:34.940752 6524 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ab01063-2737-4b29-ade3-ef9cc00ab5e1-home\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:34.942315 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:34.942294 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ab01063-2737-4b29-ade3-ef9cc00ab5e1-tls-certs" pod "1ab01063-2737-4b29-ade3-ef9cc00ab5e1" (UID: "1ab01063-2737-4b29-ade3-ef9cc00ab5e1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 03 21:59:34.942768 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:34.942744 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ab01063-2737-4b29-ade3-ef9cc00ab5e1-dshm" pod "1ab01063-2737-4b29-ade3-ef9cc00ab5e1" (UID: "1ab01063-2737-4b29-ade3-ef9cc00ab5e1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 21:59:34.942863 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:34.942769 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ab01063-2737-4b29-ade3-ef9cc00ab5e1-kube-api-access-cj2kh" pod "1ab01063-2737-4b29-ade3-ef9cc00ab5e1" (UID: "1ab01063-2737-4b29-ade3-ef9cc00ab5e1"). InnerVolumeSpecName "kube-api-access-cj2kh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 03 21:59:35.041393 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:35.041357 6524 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ab01063-2737-4b29-ade3-ef9cc00ab5e1-tls-certs\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:35.041393 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:35.041388 6524 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ab01063-2737-4b29-ade3-ef9cc00ab5e1-dshm\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:35.041393 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:35.041399 6524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cj2kh\" (UniqueName: \"kubernetes.io/projected/1ab01063-2737-4b29-ade3-ef9cc00ab5e1-kube-api-access-cj2kh\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:35.390837 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:35.390796 6524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5c184d21-bd7n9"] Nov 03 21:59:35.391348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:35.391313 6524 cpu_manager.go:414] "containerMap: removing container" podUID="1ab01063-2737-4b29-ade3-ef9cc00ab5e1" containerName="main" Nov 03 21:59:35.391348 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:35.391335 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1ab01063-2737-4b29-ade3-ef9cc00ab5e1" containerName="main" Nov 03 21:59:35.391538 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:35.391438 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="1ab01063-2737-4b29-ade3-ef9cc00ab5e1" containerName="main" Nov 03 21:59:35.394661 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:35.394639 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5c184d21-bd7n9" Nov 03 21:59:35.405768 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:35.405607 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5c184d21-bd7n9"] Nov 03 21:59:35.537463 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:35.537426 6524 generic.go:356] "Generic (PLEG): container finished" podID="1ab01063-2737-4b29-ade3-ef9cc00ab5e1" containerID="fe389a2e12e1ea5918c87e8a762d1c1f8f166b701ce9d3b273797bd9e6d30358" exitCode=0 Nov 03 21:59:35.537634 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:35.537476 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf" Nov 03 21:59:35.537634 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:35.537515 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf" event={"ID":"1ab01063-2737-4b29-ade3-ef9cc00ab5e1","Type":"ContainerDied","Data":"fe389a2e12e1ea5918c87e8a762d1c1f8f166b701ce9d3b273797bd9e6d30358"} Nov 03 21:59:35.537634 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:35.537562 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf" event={"ID":"1ab01063-2737-4b29-ade3-ef9cc00ab5e1","Type":"ContainerDied","Data":"8b0fcd55ef182e0b222c24cfcf870215b9f5d910de993bfe370700c7fb15c7dc"} Nov 03 21:59:35.537634 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:35.537584 6524 scope.go:122] "RemoveContainer" containerID="fe389a2e12e1ea5918c87e8a762d1c1f8f166b701ce9d3b273797bd9e6d30358" Nov 03 21:59:35.539216 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:35.539190 6524 generic.go:356] "Generic (PLEG): container finished" podID="fdad2bdf-ab1d-4800-9557-3e9b88953a80" containerID="7945b79fc07623b834f9549b92ffb9077c6db865b73179e2fcdf31c8723fa1a2" exitCode=0 Nov 03 21:59:35.539309 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:35.539242 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e85a4dd4-zn26z" event={"ID":"fdad2bdf-ab1d-4800-9557-3e9b88953a80","Type":"ContainerDied","Data":"7945b79fc07623b834f9549b92ffb9077c6db865b73179e2fcdf31c8723fa1a2"} Nov 03 21:59:35.547016 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:35.546998 6524 scope.go:122] "RemoveContainer" containerID="fe389a2e12e1ea5918c87e8a762d1c1f8f166b701ce9d3b273797bd9e6d30358" Nov 03 21:59:35.547323 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:59:35.547299 6524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe389a2e12e1ea5918c87e8a762d1c1f8f166b701ce9d3b273797bd9e6d30358\": container with ID starting with fe389a2e12e1ea5918c87e8a762d1c1f8f166b701ce9d3b273797bd9e6d30358 not found: ID does not exist" containerID="fe389a2e12e1ea5918c87e8a762d1c1f8f166b701ce9d3b273797bd9e6d30358" Nov 03 21:59:35.547415 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:35.547328 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe389a2e12e1ea5918c87e8a762d1c1f8f166b701ce9d3b273797bd9e6d30358"} err="failed to get container status \"fe389a2e12e1ea5918c87e8a762d1c1f8f166b701ce9d3b273797bd9e6d30358\": rpc error: code = NotFound desc = could not find container \"fe389a2e12e1ea5918c87e8a762d1c1f8f166b701ce9d3b273797bd9e6d30358\": container with ID starting with fe389a2e12e1ea5918c87e8a762d1c1f8f166b701ce9d3b273797bd9e6d30358 not found: ID does not exist" Nov 03 21:59:35.547463 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:35.547424 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/45474bce-683e-4cee-9b5b-0356ed6849f2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5c184d21-bd7n9\" (UID: \"45474bce-683e-4cee-9b5b-0356ed6849f2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5c184d21-bd7n9" Nov 03 21:59:35.547564 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:35.547546 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/45474bce-683e-4cee-9b5b-0356ed6849f2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5c184d21-bd7n9\" (UID: \"45474bce-683e-4cee-9b5b-0356ed6849f2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5c184d21-bd7n9" Nov 03 21:59:35.547609 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:35.547597 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4rt6\" (UniqueName: \"kubernetes.io/projected/45474bce-683e-4cee-9b5b-0356ed6849f2-kube-api-access-n4rt6\") pod \"maas-api-revoke-sub-5c184d21-bd7n9\" (UID: \"45474bce-683e-4cee-9b5b-0356ed6849f2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5c184d21-bd7n9" Nov 03 21:59:35.578941 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:35.578910 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf"] Nov 03 21:59:35.584796 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:35.584753 6524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-c0caef/simulator-w0-c0caef-kserve-5967669799-xtbxf"] Nov 03 21:59:35.649286 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:35.649177 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/45474bce-683e-4cee-9b5b-0356ed6849f2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5c184d21-bd7n9\" (UID: \"45474bce-683e-4cee-9b5b-0356ed6849f2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5c184d21-bd7n9" Nov 03 21:59:35.649286 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:35.649279 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/45474bce-683e-4cee-9b5b-0356ed6849f2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5c184d21-bd7n9\" (UID: \"45474bce-683e-4cee-9b5b-0356ed6849f2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5c184d21-bd7n9" Nov 03 21:59:35.649536 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:35.649316 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4rt6\" (UniqueName: \"kubernetes.io/projected/45474bce-683e-4cee-9b5b-0356ed6849f2-kube-api-access-n4rt6\") pod \"maas-api-revoke-sub-5c184d21-bd7n9\" (UID: \"45474bce-683e-4cee-9b5b-0356ed6849f2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5c184d21-bd7n9" Nov 03 21:59:35.649940 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:35.649906 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/45474bce-683e-4cee-9b5b-0356ed6849f2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5c184d21-bd7n9\" (UID: \"45474bce-683e-4cee-9b5b-0356ed6849f2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5c184d21-bd7n9" Nov 03 21:59:35.651834 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:35.651811 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/45474bce-683e-4cee-9b5b-0356ed6849f2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5c184d21-bd7n9\" (UID: \"45474bce-683e-4cee-9b5b-0356ed6849f2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5c184d21-bd7n9" Nov 03 21:59:35.661629 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:35.661609 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4rt6\" (UniqueName: \"kubernetes.io/projected/45474bce-683e-4cee-9b5b-0356ed6849f2-kube-api-access-n4rt6\") pod \"maas-api-revoke-sub-5c184d21-bd7n9\" (UID: \"45474bce-683e-4cee-9b5b-0356ed6849f2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5c184d21-bd7n9" Nov 03 21:59:35.730145 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:35.729370 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5c184d21-bd7n9" Nov 03 21:59:36.015751 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:36.015727 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5c184d21-bd7n9"] Nov 03 21:59:36.017755 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:59:36.017726 6524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45474bce_683e_4cee_9b5b_0356ed6849f2.slice/crio-972009e80dbbe95830fa92b955dd264f7fe6bfeac92dac54147d84d43ff2719f WatchSource:0}: Error finding container 972009e80dbbe95830fa92b955dd264f7fe6bfeac92dac54147d84d43ff2719f: Status 404 returned error can't find the container with id 972009e80dbbe95830fa92b955dd264f7fe6bfeac92dac54147d84d43ff2719f Nov 03 21:59:36.079224 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:36.079185 6524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w6-8af3f3-655f44685b-wh9n2" podUID="171f367e-e35d-4444-86b4-2842b49424fb" containerName="payload-processing" containerID="cri-o://d977ae9bcb79749def02d4d045471895c08c7a6817720f02af879e2121d5970a" gracePeriod=25 Nov 03 21:59:36.082792 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:36.082754 6524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-8af3f3-7c9fcbb455-25425" podUID="2e7d9548-49de-403f-8d4e-14eb05b8b338" containerName="payload-pre-processing" containerID="cri-o://7dfa6ab0c2b15fb9851ea22605e8a6c45b019d97b5f25490e6834e84a74ea5b9" gracePeriod=25 Nov 03 21:59:36.556452 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:36.556331 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5c184d21-bd7n9" event={"ID":"45474bce-683e-4cee-9b5b-0356ed6849f2","Type":"ContainerStarted","Data":"d274abdfe88a42bab5147b02de76c56d3d964430a887971f1ab1365436e955aa"} Nov 03 21:59:36.556452 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:36.556389 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5c184d21-bd7n9" event={"ID":"45474bce-683e-4cee-9b5b-0356ed6849f2","Type":"ContainerStarted","Data":"972009e80dbbe95830fa92b955dd264f7fe6bfeac92dac54147d84d43ff2719f"} Nov 03 21:59:36.576503 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:36.576454 6524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5c184d21-bd7n9" podStartSLOduration=1.576438786 podStartE2EDuration="1.576438786s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:59:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:59:36.574635261 +0000 UTC m=+821.926962049" watchObservedRunningTime="2026-11-03 21:59:36.576438786 +0000 UTC m=+821.928765591" Nov 03 21:59:36.676672 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:36.676646 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e85a4dd4-zn26z" Nov 03 21:59:36.760746 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:36.760707 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fdad2bdf-ab1d-4800-9557-3e9b88953a80-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fdad2bdf-ab1d-4800-9557-3e9b88953a80-maas-api-openshift-ca\") pod \"fdad2bdf-ab1d-4800-9557-3e9b88953a80\" (UID: \"fdad2bdf-ab1d-4800-9557-3e9b88953a80\") " Nov 03 21:59:36.760920 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:36.760759 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fdad2bdf-ab1d-4800-9557-3e9b88953a80-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fdad2bdf-ab1d-4800-9557-3e9b88953a80-maas-api-xks-ca\") pod \"fdad2bdf-ab1d-4800-9557-3e9b88953a80\" (UID: \"fdad2bdf-ab1d-4800-9557-3e9b88953a80\") " Nov 03 21:59:36.760920 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:36.760807 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fdad2bdf-ab1d-4800-9557-3e9b88953a80-kube-api-access-2862v\" (UniqueName: \"kubernetes.io/projected/fdad2bdf-ab1d-4800-9557-3e9b88953a80-kube-api-access-2862v\") pod \"fdad2bdf-ab1d-4800-9557-3e9b88953a80\" (UID: \"fdad2bdf-ab1d-4800-9557-3e9b88953a80\") " Nov 03 21:59:36.761450 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:36.761149 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fdad2bdf-ab1d-4800-9557-3e9b88953a80-maas-api-openshift-ca" pod "fdad2bdf-ab1d-4800-9557-3e9b88953a80" (UID: "fdad2bdf-ab1d-4800-9557-3e9b88953a80"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 03 21:59:36.761610 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:36.761589 6524 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fdad2bdf-ab1d-4800-9557-3e9b88953a80-maas-api-openshift-ca\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:36.763183 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:36.763157 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fdad2bdf-ab1d-4800-9557-3e9b88953a80-maas-api-xks-ca" pod "fdad2bdf-ab1d-4800-9557-3e9b88953a80" (UID: "fdad2bdf-ab1d-4800-9557-3e9b88953a80"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 03 21:59:36.763295 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:36.763273 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdad2bdf-ab1d-4800-9557-3e9b88953a80-kube-api-access-2862v" pod "fdad2bdf-ab1d-4800-9557-3e9b88953a80" (UID: "fdad2bdf-ab1d-4800-9557-3e9b88953a80"). InnerVolumeSpecName "kube-api-access-2862v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 03 21:59:36.768060 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:36.768034 6524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1ab01063-2737-4b29-ade3-ef9cc00ab5e1" path="/var/lib/kubelet/pods/1ab01063-2737-4b29-ade3-ef9cc00ab5e1/volumes" Nov 03 21:59:36.862303 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:36.862271 6524 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fdad2bdf-ab1d-4800-9557-3e9b88953a80-maas-api-xks-ca\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:36.862303 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:36.862303 6524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2862v\" (UniqueName: \"kubernetes.io/projected/fdad2bdf-ab1d-4800-9557-3e9b88953a80-kube-api-access-2862v\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:37.562191 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:37.562163 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e85a4dd4-zn26z" Nov 03 21:59:37.562639 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:37.562161 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e85a4dd4-zn26z" event={"ID":"fdad2bdf-ab1d-4800-9557-3e9b88953a80","Type":"ContainerDied","Data":"666813f28d51773c11a25ccf93b7692e38461cb20a8fb009d1de0a8b6b9d7696"} Nov 03 21:59:37.562639 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:37.562280 6524 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="666813f28d51773c11a25ccf93b7692e38461cb20a8fb009d1de0a8b6b9d7696" Nov 03 21:59:37.563986 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:37.563962 6524 generic.go:356] "Generic (PLEG): container finished" podID="45474bce-683e-4cee-9b5b-0356ed6849f2" containerID="d274abdfe88a42bab5147b02de76c56d3d964430a887971f1ab1365436e955aa" exitCode=0 Nov 03 21:59:37.564139 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:37.564032 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5c184d21-bd7n9" event={"ID":"45474bce-683e-4cee-9b5b-0356ed6849f2","Type":"ContainerDied","Data":"d274abdfe88a42bab5147b02de76c56d3d964430a887971f1ab1365436e955aa"} Nov 03 21:59:38.702451 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:38.702421 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5c184d21-bd7n9" Nov 03 21:59:38.778402 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:38.778358 6524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j" podUID="7df4ae2c-a36f-4933-9e10-1f712fb47595" containerName="main" containerID="cri-o://132404f1eaaac3fcb29279a843e96df653ca4ffafddfab697c556919b5f4a519" gracePeriod=45 Nov 03 21:59:38.782038 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:38.782009 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/45474bce-683e-4cee-9b5b-0356ed6849f2-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/45474bce-683e-4cee-9b5b-0356ed6849f2-maas-api-xks-ca\") pod \"45474bce-683e-4cee-9b5b-0356ed6849f2\" (UID: \"45474bce-683e-4cee-9b5b-0356ed6849f2\") " Nov 03 21:59:38.782176 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:38.782105 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/45474bce-683e-4cee-9b5b-0356ed6849f2-kube-api-access-n4rt6\" (UniqueName: \"kubernetes.io/projected/45474bce-683e-4cee-9b5b-0356ed6849f2-kube-api-access-n4rt6\") pod \"45474bce-683e-4cee-9b5b-0356ed6849f2\" (UID: \"45474bce-683e-4cee-9b5b-0356ed6849f2\") " Nov 03 21:59:38.782255 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:38.782203 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/45474bce-683e-4cee-9b5b-0356ed6849f2-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/45474bce-683e-4cee-9b5b-0356ed6849f2-maas-api-openshift-ca\") pod \"45474bce-683e-4cee-9b5b-0356ed6849f2\" (UID: \"45474bce-683e-4cee-9b5b-0356ed6849f2\") " Nov 03 21:59:38.782591 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:38.782559 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45474bce-683e-4cee-9b5b-0356ed6849f2-maas-api-openshift-ca" pod "45474bce-683e-4cee-9b5b-0356ed6849f2" (UID: "45474bce-683e-4cee-9b5b-0356ed6849f2"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 03 21:59:38.782832 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:38.782815 6524 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/45474bce-683e-4cee-9b5b-0356ed6849f2-maas-api-openshift-ca\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:38.784270 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:38.784248 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45474bce-683e-4cee-9b5b-0356ed6849f2-maas-api-xks-ca" pod "45474bce-683e-4cee-9b5b-0356ed6849f2" (UID: "45474bce-683e-4cee-9b5b-0356ed6849f2"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 03 21:59:38.784750 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:38.784727 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45474bce-683e-4cee-9b5b-0356ed6849f2-kube-api-access-n4rt6" pod "45474bce-683e-4cee-9b5b-0356ed6849f2" (UID: "45474bce-683e-4cee-9b5b-0356ed6849f2"). InnerVolumeSpecName "kube-api-access-n4rt6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 03 21:59:38.883786 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:38.883749 6524 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/45474bce-683e-4cee-9b5b-0356ed6849f2-maas-api-xks-ca\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:38.883786 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:38.883783 6524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n4rt6\" (UniqueName: \"kubernetes.io/projected/45474bce-683e-4cee-9b5b-0356ed6849f2-kube-api-access-n4rt6\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:39.051968 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:39.051940 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j" Nov 03 21:59:39.090907 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:39.090868 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7df4ae2c-a36f-4933-9e10-1f712fb47595-home" pod "7df4ae2c-a36f-4933-9e10-1f712fb47595" (UID: "7df4ae2c-a36f-4933-9e10-1f712fb47595"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 21:59:39.091100 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:39.090925 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7df4ae2c-a36f-4933-9e10-1f712fb47595-home\" (UniqueName: \"kubernetes.io/empty-dir/7df4ae2c-a36f-4933-9e10-1f712fb47595-home\") pod \"7df4ae2c-a36f-4933-9e10-1f712fb47595\" (UID: \"7df4ae2c-a36f-4933-9e10-1f712fb47595\") " Nov 03 21:59:39.091100 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:39.090970 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7df4ae2c-a36f-4933-9e10-1f712fb47595-tls-certs\" (UniqueName: \"kubernetes.io/secret/7df4ae2c-a36f-4933-9e10-1f712fb47595-tls-certs\") pod \"7df4ae2c-a36f-4933-9e10-1f712fb47595\" (UID: \"7df4ae2c-a36f-4933-9e10-1f712fb47595\") " Nov 03 21:59:39.091100 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:39.091028 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7df4ae2c-a36f-4933-9e10-1f712fb47595-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7df4ae2c-a36f-4933-9e10-1f712fb47595-tmp-dir\") pod \"7df4ae2c-a36f-4933-9e10-1f712fb47595\" (UID: \"7df4ae2c-a36f-4933-9e10-1f712fb47595\") " Nov 03 21:59:39.091293 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:39.091170 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7df4ae2c-a36f-4933-9e10-1f712fb47595-kube-api-access-m98tz\" (UniqueName: \"kubernetes.io/projected/7df4ae2c-a36f-4933-9e10-1f712fb47595-kube-api-access-m98tz\") pod \"7df4ae2c-a36f-4933-9e10-1f712fb47595\" (UID: \"7df4ae2c-a36f-4933-9e10-1f712fb47595\") " Nov 03 21:59:39.091293 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:39.091234 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7df4ae2c-a36f-4933-9e10-1f712fb47595-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7df4ae2c-a36f-4933-9e10-1f712fb47595-model-cache\") pod \"7df4ae2c-a36f-4933-9e10-1f712fb47595\" (UID: \"7df4ae2c-a36f-4933-9e10-1f712fb47595\") " Nov 03 21:59:39.091293 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:39.091288 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7df4ae2c-a36f-4933-9e10-1f712fb47595-dshm\" (UniqueName: \"kubernetes.io/empty-dir/7df4ae2c-a36f-4933-9e10-1f712fb47595-dshm\") pod \"7df4ae2c-a36f-4933-9e10-1f712fb47595\" (UID: \"7df4ae2c-a36f-4933-9e10-1f712fb47595\") " Nov 03 21:59:39.091627 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:39.091598 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7df4ae2c-a36f-4933-9e10-1f712fb47595-tmp-dir" pod "7df4ae2c-a36f-4933-9e10-1f712fb47595" (UID: "7df4ae2c-a36f-4933-9e10-1f712fb47595"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 21:59:39.091898 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:39.091854 6524 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7df4ae2c-a36f-4933-9e10-1f712fb47595-home\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:39.091898 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:39.091879 6524 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7df4ae2c-a36f-4933-9e10-1f712fb47595-tmp-dir\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:39.092512 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:39.092484 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7df4ae2c-a36f-4933-9e10-1f712fb47595-model-cache" pod "7df4ae2c-a36f-4933-9e10-1f712fb47595" (UID: "7df4ae2c-a36f-4933-9e10-1f712fb47595"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 21:59:39.094848 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:39.094820 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7df4ae2c-a36f-4933-9e10-1f712fb47595-tls-certs" pod "7df4ae2c-a36f-4933-9e10-1f712fb47595" (UID: "7df4ae2c-a36f-4933-9e10-1f712fb47595"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 03 21:59:39.095087 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:39.095061 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7df4ae2c-a36f-4933-9e10-1f712fb47595-dshm" pod "7df4ae2c-a36f-4933-9e10-1f712fb47595" (UID: "7df4ae2c-a36f-4933-9e10-1f712fb47595"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 21:59:39.095087 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:39.095072 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7df4ae2c-a36f-4933-9e10-1f712fb47595-kube-api-access-m98tz" pod "7df4ae2c-a36f-4933-9e10-1f712fb47595" (UID: "7df4ae2c-a36f-4933-9e10-1f712fb47595"). InnerVolumeSpecName "kube-api-access-m98tz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 03 21:59:39.193791 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:39.193711 6524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m98tz\" (UniqueName: \"kubernetes.io/projected/7df4ae2c-a36f-4933-9e10-1f712fb47595-kube-api-access-m98tz\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:39.193791 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:39.193750 6524 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7df4ae2c-a36f-4933-9e10-1f712fb47595-model-cache\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:39.193791 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:39.193767 6524 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7df4ae2c-a36f-4933-9e10-1f712fb47595-dshm\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:39.193791 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:39.193780 6524 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7df4ae2c-a36f-4933-9e10-1f712fb47595-tls-certs\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:39.626689 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:39.619673 6524 generic.go:356] "Generic (PLEG): container finished" podID="7df4ae2c-a36f-4933-9e10-1f712fb47595" containerID="132404f1eaaac3fcb29279a843e96df653ca4ffafddfab697c556919b5f4a519" exitCode=0 Nov 03 21:59:39.626689 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:39.619841 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j" Nov 03 21:59:39.626689 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:39.622762 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j" event={"ID":"7df4ae2c-a36f-4933-9e10-1f712fb47595","Type":"ContainerDied","Data":"132404f1eaaac3fcb29279a843e96df653ca4ffafddfab697c556919b5f4a519"} Nov 03 21:59:39.626689 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:39.622832 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j" event={"ID":"7df4ae2c-a36f-4933-9e10-1f712fb47595","Type":"ContainerDied","Data":"a14989a22df6d2af6f482433a0fef70df1da24b1dc3302fa134d1bc17a6c77b5"} Nov 03 21:59:39.626689 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:39.622855 6524 scope.go:122] "RemoveContainer" containerID="132404f1eaaac3fcb29279a843e96df653ca4ffafddfab697c556919b5f4a519" Nov 03 21:59:39.626689 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:39.624875 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5c184d21-bd7n9" Nov 03 21:59:39.626689 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:39.624932 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5c184d21-bd7n9" event={"ID":"45474bce-683e-4cee-9b5b-0356ed6849f2","Type":"ContainerDied","Data":"972009e80dbbe95830fa92b955dd264f7fe6bfeac92dac54147d84d43ff2719f"} Nov 03 21:59:39.626689 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:39.625001 6524 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="972009e80dbbe95830fa92b955dd264f7fe6bfeac92dac54147d84d43ff2719f" Nov 03 21:59:39.650320 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:39.649883 6524 scope.go:122] "RemoveContainer" containerID="132404f1eaaac3fcb29279a843e96df653ca4ffafddfab697c556919b5f4a519" Nov 03 21:59:39.671172 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:59:39.666458 6524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"132404f1eaaac3fcb29279a843e96df653ca4ffafddfab697c556919b5f4a519\": container with ID starting with 132404f1eaaac3fcb29279a843e96df653ca4ffafddfab697c556919b5f4a519 not found: ID does not exist" containerID="132404f1eaaac3fcb29279a843e96df653ca4ffafddfab697c556919b5f4a519" Nov 03 21:59:39.671172 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:39.666512 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"132404f1eaaac3fcb29279a843e96df653ca4ffafddfab697c556919b5f4a519"} err="failed to get container status \"132404f1eaaac3fcb29279a843e96df653ca4ffafddfab697c556919b5f4a519\": rpc error: code = NotFound desc = could not find container \"132404f1eaaac3fcb29279a843e96df653ca4ffafddfab697c556919b5f4a519\": container with ID starting with 132404f1eaaac3fcb29279a843e96df653ca4ffafddfab697c556919b5f4a519 not found: ID does not exist" Nov 03 21:59:39.704356 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:39.704324 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j"] Nov 03 21:59:39.714664 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:39.714620 6524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-f4fb1b/premium-w1-f4fb1b-kserve-5895cf468b-xg88j"] Nov 03 21:59:40.767864 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:40.767832 6524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7df4ae2c-a36f-4933-9e10-1f712fb47595" path="/var/lib/kubelet/pods/7df4ae2c-a36f-4933-9e10-1f712fb47595/volumes" Nov 03 21:59:43.582904 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:43.582835 6524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z" podUID="e0ef1acd-e793-4ea1-b89c-924d84be20bc" containerName="main" containerID="cri-o://f494e9cda746fcb2a738c0f76e26e6e224be24694d21650def57269eb2c68d7c" gracePeriod=45 Nov 03 21:59:43.612090 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:43.612049 6524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z" podUID="e0ef1acd-e793-4ea1-b89c-924d84be20bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.33:8000/ready\": dial tcp 10.133.0.33:8000: connect: connection refused" Nov 03 21:59:43.864593 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:43.864570 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z" Nov 03 21:59:43.960450 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:43.960423 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e0ef1acd-e793-4ea1-b89c-924d84be20bc-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0ef1acd-e793-4ea1-b89c-924d84be20bc-dshm\") pod \"e0ef1acd-e793-4ea1-b89c-924d84be20bc\" (UID: \"e0ef1acd-e793-4ea1-b89c-924d84be20bc\") " Nov 03 21:59:43.960597 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:43.960466 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e0ef1acd-e793-4ea1-b89c-924d84be20bc-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0ef1acd-e793-4ea1-b89c-924d84be20bc-model-cache\") pod \"e0ef1acd-e793-4ea1-b89c-924d84be20bc\" (UID: \"e0ef1acd-e793-4ea1-b89c-924d84be20bc\") " Nov 03 21:59:43.960597 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:43.960488 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e0ef1acd-e793-4ea1-b89c-924d84be20bc-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0ef1acd-e793-4ea1-b89c-924d84be20bc-tmp-dir\") pod \"e0ef1acd-e793-4ea1-b89c-924d84be20bc\" (UID: \"e0ef1acd-e793-4ea1-b89c-924d84be20bc\") " Nov 03 21:59:43.960597 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:43.960506 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e0ef1acd-e793-4ea1-b89c-924d84be20bc-tls-certs\" (UniqueName: \"kubernetes.io/secret/e0ef1acd-e793-4ea1-b89c-924d84be20bc-tls-certs\") pod \"e0ef1acd-e793-4ea1-b89c-924d84be20bc\" (UID: \"e0ef1acd-e793-4ea1-b89c-924d84be20bc\") " Nov 03 21:59:43.960597 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:43.960563 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e0ef1acd-e793-4ea1-b89c-924d84be20bc-home\" (UniqueName: \"kubernetes.io/empty-dir/e0ef1acd-e793-4ea1-b89c-924d84be20bc-home\") pod \"e0ef1acd-e793-4ea1-b89c-924d84be20bc\" (UID: \"e0ef1acd-e793-4ea1-b89c-924d84be20bc\") " Nov 03 21:59:43.960810 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:43.960613 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e0ef1acd-e793-4ea1-b89c-924d84be20bc-kube-api-access-ww8t9\" (UniqueName: \"kubernetes.io/projected/e0ef1acd-e793-4ea1-b89c-924d84be20bc-kube-api-access-ww8t9\") pod \"e0ef1acd-e793-4ea1-b89c-924d84be20bc\" (UID: \"e0ef1acd-e793-4ea1-b89c-924d84be20bc\") " Nov 03 21:59:43.960810 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:43.960747 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0ef1acd-e793-4ea1-b89c-924d84be20bc-tmp-dir" pod "e0ef1acd-e793-4ea1-b89c-924d84be20bc" (UID: "e0ef1acd-e793-4ea1-b89c-924d84be20bc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 21:59:43.960810 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:43.960758 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0ef1acd-e793-4ea1-b89c-924d84be20bc-model-cache" pod "e0ef1acd-e793-4ea1-b89c-924d84be20bc" (UID: "e0ef1acd-e793-4ea1-b89c-924d84be20bc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 21:59:43.960954 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:43.960864 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0ef1acd-e793-4ea1-b89c-924d84be20bc-home" pod "e0ef1acd-e793-4ea1-b89c-924d84be20bc" (UID: "e0ef1acd-e793-4ea1-b89c-924d84be20bc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 21:59:43.961032 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:43.961014 6524 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0ef1acd-e793-4ea1-b89c-924d84be20bc-home\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:43.961082 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:43.961038 6524 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0ef1acd-e793-4ea1-b89c-924d84be20bc-model-cache\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:43.961082 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:43.961055 6524 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0ef1acd-e793-4ea1-b89c-924d84be20bc-tmp-dir\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:43.962722 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:43.962698 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0ef1acd-e793-4ea1-b89c-924d84be20bc-kube-api-access-ww8t9" pod "e0ef1acd-e793-4ea1-b89c-924d84be20bc" (UID: "e0ef1acd-e793-4ea1-b89c-924d84be20bc"). InnerVolumeSpecName "kube-api-access-ww8t9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 03 21:59:43.962722 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:43.962709 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0ef1acd-e793-4ea1-b89c-924d84be20bc-dshm" pod "e0ef1acd-e793-4ea1-b89c-924d84be20bc" (UID: "e0ef1acd-e793-4ea1-b89c-924d84be20bc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 21:59:43.962862 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:43.962758 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0ef1acd-e793-4ea1-b89c-924d84be20bc-tls-certs" pod "e0ef1acd-e793-4ea1-b89c-924d84be20bc" (UID: "e0ef1acd-e793-4ea1-b89c-924d84be20bc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 03 21:59:44.061628 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:44.061589 6524 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0ef1acd-e793-4ea1-b89c-924d84be20bc-dshm\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:44.061628 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:44.061620 6524 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0ef1acd-e793-4ea1-b89c-924d84be20bc-tls-certs\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:44.061628 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:44.061630 6524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ww8t9\" (UniqueName: \"kubernetes.io/projected/e0ef1acd-e793-4ea1-b89c-924d84be20bc-kube-api-access-ww8t9\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:44.694146 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:44.690602 6524 generic.go:356] "Generic (PLEG): container finished" podID="e0ef1acd-e793-4ea1-b89c-924d84be20bc" containerID="f494e9cda746fcb2a738c0f76e26e6e224be24694d21650def57269eb2c68d7c" exitCode=0 Nov 03 21:59:44.694146 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:44.690704 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z" event={"ID":"e0ef1acd-e793-4ea1-b89c-924d84be20bc","Type":"ContainerDied","Data":"f494e9cda746fcb2a738c0f76e26e6e224be24694d21650def57269eb2c68d7c"} Nov 03 21:59:44.694146 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:44.690745 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z" event={"ID":"e0ef1acd-e793-4ea1-b89c-924d84be20bc","Type":"ContainerDied","Data":"2f45ac3107785dc2947a9a488d05fd4ad97a36dc59c67570a3c01516f8c7192d"} Nov 03 21:59:44.694146 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:44.690771 6524 scope.go:122] "RemoveContainer" containerID="f494e9cda746fcb2a738c0f76e26e6e224be24694d21650def57269eb2c68d7c" Nov 03 21:59:44.694146 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:44.690879 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z" Nov 03 21:59:44.724384 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:44.724356 6524 scope.go:122] "RemoveContainer" containerID="f494e9cda746fcb2a738c0f76e26e6e224be24694d21650def57269eb2c68d7c" Nov 03 21:59:44.724733 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:59:44.724705 6524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f494e9cda746fcb2a738c0f76e26e6e224be24694d21650def57269eb2c68d7c\": container with ID starting with f494e9cda746fcb2a738c0f76e26e6e224be24694d21650def57269eb2c68d7c not found: ID does not exist" containerID="f494e9cda746fcb2a738c0f76e26e6e224be24694d21650def57269eb2c68d7c" Nov 03 21:59:44.724779 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:44.724748 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f494e9cda746fcb2a738c0f76e26e6e224be24694d21650def57269eb2c68d7c"} err="failed to get container status \"f494e9cda746fcb2a738c0f76e26e6e224be24694d21650def57269eb2c68d7c\": rpc error: code = NotFound desc = could not find container \"f494e9cda746fcb2a738c0f76e26e6e224be24694d21650def57269eb2c68d7c\": container with ID starting with f494e9cda746fcb2a738c0f76e26e6e224be24694d21650def57269eb2c68d7c not found: ID does not exist" Nov 03 21:59:44.728148 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:44.728106 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z"] Nov 03 21:59:44.736112 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:44.736077 6524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-02e8ab/premium-w3-02e8ab-kserve-7d4c6c68dc-8w22z"] Nov 03 21:59:44.770513 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:44.770469 6524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e0ef1acd-e793-4ea1-b89c-924d84be20bc" path="/var/lib/kubelet/pods/e0ef1acd-e793-4ea1-b89c-924d84be20bc/volumes" Nov 03 21:59:44.785831 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:44.785800 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-f4fb1b-gxdmt"] Nov 03 21:59:44.790169 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:44.790140 6524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-f4fb1b-gxdmt"] Nov 03 21:59:45.198615 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:45.198572 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv"] Nov 03 21:59:45.496858 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:45.496760 6524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w6-8af3f3-655f44685b-wh9n2" podUID="171f367e-e35d-4444-86b4-2842b49424fb" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Nov 03 21:59:45.496858 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:45.496781 6524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-8af3f3-7c9fcbb455-25425" podUID="2e7d9548-49de-403f-8d4e-14eb05b8b338" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Nov 03 21:59:46.774649 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:46.774596 6524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7a080818-a714-41a6-b2e1-b433512dc395" path="/var/lib/kubelet/pods/7a080818-a714-41a6-b2e1-b433512dc395/volumes" Nov 03 21:59:47.297162 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:47.283905 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-8af3f3-8bcd46f45-qbvvd"] Nov 03 21:59:47.297162 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:47.284179 6524 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-8af3f3-8bcd46f45-qbvvd" podUID="1818211e-c9ac-4329-ad91-84825e22a424" containerName="maas-api" containerID="cri-o://5fa1571281442d07a73e4be32ebdc4ca451e9ca07f50d816aa8f33939d2f5d5f" gracePeriod=30 Nov 03 21:59:47.596194 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:47.595530 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-8af3f3-8bcd46f45-qbvvd" Nov 03 21:59:47.718202 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:47.714604 6524 generic.go:356] "Generic (PLEG): container finished" podID="1818211e-c9ac-4329-ad91-84825e22a424" containerID="5fa1571281442d07a73e4be32ebdc4ca451e9ca07f50d816aa8f33939d2f5d5f" exitCode=0 Nov 03 21:59:47.718202 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:47.714784 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-8af3f3-8bcd46f45-qbvvd" event={"ID":"1818211e-c9ac-4329-ad91-84825e22a424","Type":"ContainerDied","Data":"5fa1571281442d07a73e4be32ebdc4ca451e9ca07f50d816aa8f33939d2f5d5f"} Nov 03 21:59:47.718202 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:47.714837 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-8af3f3-8bcd46f45-qbvvd" event={"ID":"1818211e-c9ac-4329-ad91-84825e22a424","Type":"ContainerDied","Data":"2a30ce57d00c63020a7d224d4f6a5157b0e955038fcfc08b2836b796b750c4cb"} Nov 03 21:59:47.718202 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:47.714868 6524 scope.go:122] "RemoveContainer" containerID="5fa1571281442d07a73e4be32ebdc4ca451e9ca07f50d816aa8f33939d2f5d5f" Nov 03 21:59:47.718202 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:47.715028 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-8af3f3-8bcd46f45-qbvvd" Nov 03 21:59:47.770182 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:47.719451 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1818211e-c9ac-4329-ad91-84825e22a424-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1818211e-c9ac-4329-ad91-84825e22a424-metrics-certs\") pod \"1818211e-c9ac-4329-ad91-84825e22a424\" (UID: \"1818211e-c9ac-4329-ad91-84825e22a424\") " Nov 03 21:59:47.770182 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:47.719550 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1818211e-c9ac-4329-ad91-84825e22a424-kube-api-access-drb4t\" (UniqueName: \"kubernetes.io/projected/1818211e-c9ac-4329-ad91-84825e22a424-kube-api-access-drb4t\") pod \"1818211e-c9ac-4329-ad91-84825e22a424\" (UID: \"1818211e-c9ac-4329-ad91-84825e22a424\") " Nov 03 21:59:47.770182 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:47.719677 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1818211e-c9ac-4329-ad91-84825e22a424-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1818211e-c9ac-4329-ad91-84825e22a424-maas-api-tls\") pod \"1818211e-c9ac-4329-ad91-84825e22a424\" (UID: \"1818211e-c9ac-4329-ad91-84825e22a424\") " Nov 03 21:59:47.770182 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:47.719870 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1818211e-c9ac-4329-ad91-84825e22a424-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1818211e-c9ac-4329-ad91-84825e22a424-openshift-service-ca\") pod \"1818211e-c9ac-4329-ad91-84825e22a424\" (UID: \"1818211e-c9ac-4329-ad91-84825e22a424\") " Nov 03 21:59:47.770182 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:47.720944 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1818211e-c9ac-4329-ad91-84825e22a424-openshift-service-ca" pod "1818211e-c9ac-4329-ad91-84825e22a424" (UID: "1818211e-c9ac-4329-ad91-84825e22a424"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 03 21:59:47.770182 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:47.758193 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-02e8ab-865jn"] Nov 03 21:59:47.770182 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:47.761372 6524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-02e8ab-865jn"] Nov 03 21:59:47.781346 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:47.781305 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1818211e-c9ac-4329-ad91-84825e22a424-kube-api-access-drb4t" pod "1818211e-c9ac-4329-ad91-84825e22a424" (UID: "1818211e-c9ac-4329-ad91-84825e22a424"). InnerVolumeSpecName "kube-api-access-drb4t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 03 21:59:47.781944 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:47.781890 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1818211e-c9ac-4329-ad91-84825e22a424-metrics-certs" pod "1818211e-c9ac-4329-ad91-84825e22a424" (UID: "1818211e-c9ac-4329-ad91-84825e22a424"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 03 21:59:47.789304 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:47.788741 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1818211e-c9ac-4329-ad91-84825e22a424-maas-api-tls" pod "1818211e-c9ac-4329-ad91-84825e22a424" (UID: "1818211e-c9ac-4329-ad91-84825e22a424"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 03 21:59:47.797747 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:47.797724 6524 scope.go:122] "RemoveContainer" containerID="5fa1571281442d07a73e4be32ebdc4ca451e9ca07f50d816aa8f33939d2f5d5f" Nov 03 21:59:47.798352 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:59:47.798204 6524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5fa1571281442d07a73e4be32ebdc4ca451e9ca07f50d816aa8f33939d2f5d5f\": container with ID starting with 5fa1571281442d07a73e4be32ebdc4ca451e9ca07f50d816aa8f33939d2f5d5f not found: ID does not exist" containerID="5fa1571281442d07a73e4be32ebdc4ca451e9ca07f50d816aa8f33939d2f5d5f" Nov 03 21:59:47.798352 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:47.798236 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fa1571281442d07a73e4be32ebdc4ca451e9ca07f50d816aa8f33939d2f5d5f"} err="failed to get container status \"5fa1571281442d07a73e4be32ebdc4ca451e9ca07f50d816aa8f33939d2f5d5f\": rpc error: code = NotFound desc = could not find container \"5fa1571281442d07a73e4be32ebdc4ca451e9ca07f50d816aa8f33939d2f5d5f\": container with ID starting with 5fa1571281442d07a73e4be32ebdc4ca451e9ca07f50d816aa8f33939d2f5d5f not found: ID does not exist" Nov 03 21:59:47.821971 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:47.821888 6524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1818211e-c9ac-4329-ad91-84825e22a424-maas-api-tls\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:47.821971 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:47.821917 6524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1818211e-c9ac-4329-ad91-84825e22a424-openshift-service-ca\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:47.821971 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:47.821928 6524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1818211e-c9ac-4329-ad91-84825e22a424-metrics-certs\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:47.821971 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:47.821939 6524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drb4t\" (UniqueName: \"kubernetes.io/projected/1818211e-c9ac-4329-ad91-84825e22a424-kube-api-access-drb4t\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:48.039066 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:48.039028 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-8af3f3-8bcd46f45-qbvvd"] Nov 03 21:59:48.043540 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:48.043511 6524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-8af3f3-8bcd46f45-qbvvd"] Nov 03 21:59:48.824160 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:48.812556 6524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1818211e-c9ac-4329-ad91-84825e22a424" path="/var/lib/kubelet/pods/1818211e-c9ac-4329-ad91-84825e22a424/volumes" Nov 03 21:59:48.825627 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:48.825594 6524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3a50052d-56d1-451b-8762-1f81880a4406" path="/var/lib/kubelet/pods/3a50052d-56d1-451b-8762-1f81880a4406/volumes" Nov 03 21:59:50.190056 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:50.190016 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j"] Nov 03 21:59:50.190577 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:50.190330 6524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" podUID="82c4e70e-4765-48d3-80d3-8e0311abfda7" containerName="istio-proxy" containerID="cri-o://a3cc8b56794c604d5e10892d7544628cc525e88f823336a6f11c3011e76cb3cc" gracePeriod=30 Nov 03 21:59:52.050490 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:52.050443 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2"] Nov 03 21:59:52.051614 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:52.051569 6524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" podUID="7a83deb0-69e5-4c9d-a573-235be3ff17ef" containerName="istio-proxy" containerID="cri-o://640fe0bf5b0c90c23ecf2082960fe45c8714919d32574ca3ee7751064084e229" gracePeriod=30 Nov 03 21:59:52.294155 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:52.287826 6524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-97163ab9-7t5ft"] Nov 03 21:59:52.294155 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:52.288508 6524 cpu_manager.go:414] "containerMap: removing container" podUID="45474bce-683e-4cee-9b5b-0356ed6849f2" containerName="revoke-keys" Nov 03 21:59:52.294155 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:52.288535 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="45474bce-683e-4cee-9b5b-0356ed6849f2" containerName="revoke-keys" Nov 03 21:59:52.294155 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:52.288551 6524 cpu_manager.go:414] "containerMap: removing container" podUID="e0ef1acd-e793-4ea1-b89c-924d84be20bc" containerName="main" Nov 03 21:59:52.294155 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:52.288560 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e0ef1acd-e793-4ea1-b89c-924d84be20bc" containerName="main" Nov 03 21:59:52.294155 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:52.288585 6524 cpu_manager.go:414] "containerMap: removing container" podUID="7df4ae2c-a36f-4933-9e10-1f712fb47595" containerName="main" Nov 03 21:59:52.294155 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:52.288593 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7df4ae2c-a36f-4933-9e10-1f712fb47595" containerName="main" Nov 03 21:59:52.294155 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:52.288606 6524 cpu_manager.go:414] "containerMap: removing container" podUID="1818211e-c9ac-4329-ad91-84825e22a424" containerName="maas-api" Nov 03 21:59:52.294155 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:52.288613 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1818211e-c9ac-4329-ad91-84825e22a424" containerName="maas-api" Nov 03 21:59:52.294155 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:52.288624 6524 cpu_manager.go:414] "containerMap: removing container" podUID="fdad2bdf-ab1d-4800-9557-3e9b88953a80" containerName="revoke-keys" Nov 03 21:59:52.294155 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:52.288630 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fdad2bdf-ab1d-4800-9557-3e9b88953a80" containerName="revoke-keys" Nov 03 21:59:52.294155 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:52.288747 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="45474bce-683e-4cee-9b5b-0356ed6849f2" containerName="revoke-keys" Nov 03 21:59:52.294155 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:52.288762 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="e0ef1acd-e793-4ea1-b89c-924d84be20bc" containerName="main" Nov 03 21:59:52.294155 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:52.288774 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="7df4ae2c-a36f-4933-9e10-1f712fb47595" containerName="main" Nov 03 21:59:52.294155 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:52.288784 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="1818211e-c9ac-4329-ad91-84825e22a424" containerName="maas-api" Nov 03 21:59:52.294155 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:52.288793 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="fdad2bdf-ab1d-4800-9557-3e9b88953a80" containerName="revoke-keys" Nov 03 21:59:52.305049 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:52.304974 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-97163ab9-7t5ft"] Nov 03 21:59:52.305352 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:52.305335 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-97163ab9-7t5ft" Nov 03 21:59:52.331651 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:52.321724 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Nov 03 21:59:52.347401 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:52.347371 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5n7j7\"" Nov 03 21:59:52.423407 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:52.423369 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvjhx\" (UniqueName: \"kubernetes.io/projected/5ff565e0-cada-4946-b535-f7cf03d8811e-kube-api-access-rvjhx\") pod \"maas-api-revoke-sub-97163ab9-7t5ft\" (UID: \"5ff565e0-cada-4946-b535-f7cf03d8811e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-97163ab9-7t5ft" Nov 03 21:59:52.423582 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:52.423455 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5ff565e0-cada-4946-b535-f7cf03d8811e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-97163ab9-7t5ft\" (UID: \"5ff565e0-cada-4946-b535-f7cf03d8811e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-97163ab9-7t5ft" Nov 03 21:59:52.423582 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:52.423517 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5ff565e0-cada-4946-b535-f7cf03d8811e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-97163ab9-7t5ft\" (UID: \"5ff565e0-cada-4946-b535-f7cf03d8811e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-97163ab9-7t5ft" Nov 03 21:59:52.525175 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:52.525119 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvjhx\" (UniqueName: \"kubernetes.io/projected/5ff565e0-cada-4946-b535-f7cf03d8811e-kube-api-access-rvjhx\") pod \"maas-api-revoke-sub-97163ab9-7t5ft\" (UID: \"5ff565e0-cada-4946-b535-f7cf03d8811e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-97163ab9-7t5ft" Nov 03 21:59:52.525359 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:52.525204 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5ff565e0-cada-4946-b535-f7cf03d8811e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-97163ab9-7t5ft\" (UID: \"5ff565e0-cada-4946-b535-f7cf03d8811e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-97163ab9-7t5ft" Nov 03 21:59:52.525359 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:52.525229 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5ff565e0-cada-4946-b535-f7cf03d8811e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-97163ab9-7t5ft\" (UID: \"5ff565e0-cada-4946-b535-f7cf03d8811e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-97163ab9-7t5ft" Nov 03 21:59:52.525848 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:52.525825 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5ff565e0-cada-4946-b535-f7cf03d8811e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-97163ab9-7t5ft\" (UID: \"5ff565e0-cada-4946-b535-f7cf03d8811e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-97163ab9-7t5ft" Nov 03 21:59:52.527588 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:52.527565 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5ff565e0-cada-4946-b535-f7cf03d8811e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-97163ab9-7t5ft\" (UID: \"5ff565e0-cada-4946-b535-f7cf03d8811e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-97163ab9-7t5ft" Nov 03 21:59:52.539267 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:52.539233 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvjhx\" (UniqueName: \"kubernetes.io/projected/5ff565e0-cada-4946-b535-f7cf03d8811e-kube-api-access-rvjhx\") pod \"maas-api-revoke-sub-97163ab9-7t5ft\" (UID: \"5ff565e0-cada-4946-b535-f7cf03d8811e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-97163ab9-7t5ft" Nov 03 21:59:52.656274 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:52.656231 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-97163ab9-7t5ft" Nov 03 21:59:52.933145 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:52.933108 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-97163ab9-7t5ft"] Nov 03 21:59:52.943637 ip-10-0-137-7 kubenswrapper[6524]: W1103 21:59:52.943530 6524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ff565e0_cada_4946_b535_f7cf03d8811e.slice/crio-f3e0c69721a8949778f8a29865475bac217233ff97e874099d2f5c8c4da0eb50 WatchSource:0}: Error finding container f3e0c69721a8949778f8a29865475bac217233ff97e874099d2f5c8c4da0eb50: Status 404 returned error can't find the container with id f3e0c69721a8949778f8a29865475bac217233ff97e874099d2f5c8c4da0eb50 Nov 03 21:59:53.179879 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:53.177968 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt"] Nov 03 21:59:53.179879 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:53.178537 6524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" podUID="efcc7e46-86ce-427c-a5d7-99878a682ac1" containerName="istio-proxy" containerID="cri-o://f00351cebd65d645d82b2ea0a3592d9e03a611ebfe205b7c9edffc54c124aaec" gracePeriod=30 Nov 03 21:59:53.771274 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:53.771226 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-97163ab9-7t5ft" event={"ID":"5ff565e0-cada-4946-b535-f7cf03d8811e","Type":"ContainerStarted","Data":"1493f3a9441a1eb789d26f04bcd0c8c9f45db5e08c621205781b1b3301136493"} Nov 03 21:59:53.771481 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:53.771330 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-97163ab9-7t5ft" event={"ID":"5ff565e0-cada-4946-b535-f7cf03d8811e","Type":"ContainerStarted","Data":"f3e0c69721a8949778f8a29865475bac217233ff97e874099d2f5c8c4da0eb50"} Nov 03 21:59:53.791735 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:53.791682 6524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-97163ab9-7t5ft" podStartSLOduration=1.791668195 podStartE2EDuration="1.791668195s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:59:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:59:53.789852407 +0000 UTC m=+839.142179209" watchObservedRunningTime="2026-11-03 21:59:53.791668195 +0000 UTC m=+839.143994998" Nov 03 21:59:54.777468 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:54.777423 6524 generic.go:356] "Generic (PLEG): container finished" podID="5ff565e0-cada-4946-b535-f7cf03d8811e" containerID="1493f3a9441a1eb789d26f04bcd0c8c9f45db5e08c621205781b1b3301136493" exitCode=0 Nov 03 21:59:54.777940 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:54.777481 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-97163ab9-7t5ft" event={"ID":"5ff565e0-cada-4946-b535-f7cf03d8811e","Type":"ContainerDied","Data":"1493f3a9441a1eb789d26f04bcd0c8c9f45db5e08c621205781b1b3301136493"} Nov 03 21:59:55.496710 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:55.496424 6524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-8af3f3-7c9fcbb455-25425" podUID="2e7d9548-49de-403f-8d4e-14eb05b8b338" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Nov 03 21:59:55.496710 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:55.496697 6524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w6-8af3f3-655f44685b-wh9n2" podUID="171f367e-e35d-4444-86b4-2842b49424fb" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Nov 03 21:59:55.641429 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:55.641392 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9"] Nov 03 21:59:55.641826 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:55.641604 6524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" podUID="779feacc-19e1-4b9e-9968-896c178c78f8" containerName="istio-proxy" containerID="cri-o://4a48814aeb26f05ca436d6300a258f42e728939c47bb85f7fbb0ae1fafa98bce" gracePeriod=30 Nov 03 21:59:55.811021 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:55.810921 6524 generic.go:356] "Generic (PLEG): container finished" podID="82c4e70e-4765-48d3-80d3-8e0311abfda7" containerID="a3cc8b56794c604d5e10892d7544628cc525e88f823336a6f11c3011e76cb3cc" exitCode=0 Nov 03 21:59:55.811529 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:55.811203 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" event={"ID":"82c4e70e-4765-48d3-80d3-8e0311abfda7","Type":"ContainerDied","Data":"a3cc8b56794c604d5e10892d7544628cc525e88f823336a6f11c3011e76cb3cc"} Nov 03 21:59:55.811529 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:55.811247 6524 scope.go:122] "RemoveContainer" containerID="c8e6e032a42781b491f35490e3e79e129f0f8aa6b69418c47760b39ef261dec2" Nov 03 21:59:55.967013 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:55.966979 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" Nov 03 21:59:56.015076 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.015047 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-97163ab9-7t5ft" Nov 03 21:59:56.070681 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.070585 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/82c4e70e-4765-48d3-80d3-8e0311abfda7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82c4e70e-4765-48d3-80d3-8e0311abfda7-credential-socket\") pod \"82c4e70e-4765-48d3-80d3-8e0311abfda7\" (UID: \"82c4e70e-4765-48d3-80d3-8e0311abfda7\") " Nov 03 21:59:56.070681 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.070627 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/82c4e70e-4765-48d3-80d3-8e0311abfda7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82c4e70e-4765-48d3-80d3-8e0311abfda7-istio-data\") pod \"82c4e70e-4765-48d3-80d3-8e0311abfda7\" (UID: \"82c4e70e-4765-48d3-80d3-8e0311abfda7\") " Nov 03 21:59:56.070681 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.070658 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/82c4e70e-4765-48d3-80d3-8e0311abfda7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82c4e70e-4765-48d3-80d3-8e0311abfda7-istiod-ca-cert\") pod \"82c4e70e-4765-48d3-80d3-8e0311abfda7\" (UID: \"82c4e70e-4765-48d3-80d3-8e0311abfda7\") " Nov 03 21:59:56.070974 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.070700 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/82c4e70e-4765-48d3-80d3-8e0311abfda7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82c4e70e-4765-48d3-80d3-8e0311abfda7-workload-socket\") pod \"82c4e70e-4765-48d3-80d3-8e0311abfda7\" (UID: \"82c4e70e-4765-48d3-80d3-8e0311abfda7\") " Nov 03 21:59:56.070974 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.070730 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/82c4e70e-4765-48d3-80d3-8e0311abfda7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82c4e70e-4765-48d3-80d3-8e0311abfda7-istio-envoy\") pod \"82c4e70e-4765-48d3-80d3-8e0311abfda7\" (UID: \"82c4e70e-4765-48d3-80d3-8e0311abfda7\") " Nov 03 21:59:56.070974 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.070802 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/82c4e70e-4765-48d3-80d3-8e0311abfda7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82c4e70e-4765-48d3-80d3-8e0311abfda7-istio-podinfo\") pod \"82c4e70e-4765-48d3-80d3-8e0311abfda7\" (UID: \"82c4e70e-4765-48d3-80d3-8e0311abfda7\") " Nov 03 21:59:56.070974 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.070828 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/82c4e70e-4765-48d3-80d3-8e0311abfda7-kube-api-access-hbh4z\" (UniqueName: \"kubernetes.io/projected/82c4e70e-4765-48d3-80d3-8e0311abfda7-kube-api-access-hbh4z\") pod \"82c4e70e-4765-48d3-80d3-8e0311abfda7\" (UID: \"82c4e70e-4765-48d3-80d3-8e0311abfda7\") " Nov 03 21:59:56.070974 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.070919 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/82c4e70e-4765-48d3-80d3-8e0311abfda7-istio-token\" (UniqueName: \"kubernetes.io/projected/82c4e70e-4765-48d3-80d3-8e0311abfda7-istio-token\") pod \"82c4e70e-4765-48d3-80d3-8e0311abfda7\" (UID: \"82c4e70e-4765-48d3-80d3-8e0311abfda7\") " Nov 03 21:59:56.070974 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.070910 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82c4e70e-4765-48d3-80d3-8e0311abfda7-istio-data" pod "82c4e70e-4765-48d3-80d3-8e0311abfda7" (UID: "82c4e70e-4765-48d3-80d3-8e0311abfda7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 21:59:56.070974 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.070963 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/82c4e70e-4765-48d3-80d3-8e0311abfda7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82c4e70e-4765-48d3-80d3-8e0311abfda7-workload-certs\") pod \"82c4e70e-4765-48d3-80d3-8e0311abfda7\" (UID: \"82c4e70e-4765-48d3-80d3-8e0311abfda7\") " Nov 03 21:59:56.070974 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.070921 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82c4e70e-4765-48d3-80d3-8e0311abfda7-credential-socket" pod "82c4e70e-4765-48d3-80d3-8e0311abfda7" (UID: "82c4e70e-4765-48d3-80d3-8e0311abfda7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 21:59:56.071410 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.071012 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82c4e70e-4765-48d3-80d3-8e0311abfda7-workload-socket" pod "82c4e70e-4765-48d3-80d3-8e0311abfda7" (UID: "82c4e70e-4765-48d3-80d3-8e0311abfda7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 21:59:56.071410 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.071229 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82c4e70e-4765-48d3-80d3-8e0311abfda7-workload-certs" pod "82c4e70e-4765-48d3-80d3-8e0311abfda7" (UID: "82c4e70e-4765-48d3-80d3-8e0311abfda7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 21:59:56.071410 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.071269 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82c4e70e-4765-48d3-80d3-8e0311abfda7-istiod-ca-cert" pod "82c4e70e-4765-48d3-80d3-8e0311abfda7" (UID: "82c4e70e-4765-48d3-80d3-8e0311abfda7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 03 21:59:56.071410 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.071394 6524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82c4e70e-4765-48d3-80d3-8e0311abfda7-workload-certs\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:56.071568 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.071412 6524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82c4e70e-4765-48d3-80d3-8e0311abfda7-credential-socket\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:56.071568 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.071429 6524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82c4e70e-4765-48d3-80d3-8e0311abfda7-istio-data\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:56.071568 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.071443 6524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82c4e70e-4765-48d3-80d3-8e0311abfda7-istiod-ca-cert\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:56.071568 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.071459 6524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82c4e70e-4765-48d3-80d3-8e0311abfda7-workload-socket\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:56.073148 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.073114 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82c4e70e-4765-48d3-80d3-8e0311abfda7-istio-token" pod "82c4e70e-4765-48d3-80d3-8e0311abfda7" (UID: "82c4e70e-4765-48d3-80d3-8e0311abfda7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 03 21:59:56.073232 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.073147 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82c4e70e-4765-48d3-80d3-8e0311abfda7-istio-envoy" pod "82c4e70e-4765-48d3-80d3-8e0311abfda7" (UID: "82c4e70e-4765-48d3-80d3-8e0311abfda7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 21:59:56.073427 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.073407 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/82c4e70e-4765-48d3-80d3-8e0311abfda7-istio-podinfo" pod "82c4e70e-4765-48d3-80d3-8e0311abfda7" (UID: "82c4e70e-4765-48d3-80d3-8e0311abfda7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 03 21:59:56.073497 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.073481 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82c4e70e-4765-48d3-80d3-8e0311abfda7-kube-api-access-hbh4z" pod "82c4e70e-4765-48d3-80d3-8e0311abfda7" (UID: "82c4e70e-4765-48d3-80d3-8e0311abfda7"). InnerVolumeSpecName "kube-api-access-hbh4z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 03 21:59:56.172000 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.171960 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5ff565e0-cada-4946-b535-f7cf03d8811e-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5ff565e0-cada-4946-b535-f7cf03d8811e-maas-api-xks-ca\") pod \"5ff565e0-cada-4946-b535-f7cf03d8811e\" (UID: \"5ff565e0-cada-4946-b535-f7cf03d8811e\") " Nov 03 21:59:56.172204 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.172014 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5ff565e0-cada-4946-b535-f7cf03d8811e-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5ff565e0-cada-4946-b535-f7cf03d8811e-maas-api-openshift-ca\") pod \"5ff565e0-cada-4946-b535-f7cf03d8811e\" (UID: \"5ff565e0-cada-4946-b535-f7cf03d8811e\") " Nov 03 21:59:56.172204 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.172140 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5ff565e0-cada-4946-b535-f7cf03d8811e-kube-api-access-rvjhx\" (UniqueName: \"kubernetes.io/projected/5ff565e0-cada-4946-b535-f7cf03d8811e-kube-api-access-rvjhx\") pod \"5ff565e0-cada-4946-b535-f7cf03d8811e\" (UID: \"5ff565e0-cada-4946-b535-f7cf03d8811e\") " Nov 03 21:59:56.172377 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.172351 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ff565e0-cada-4946-b535-f7cf03d8811e-maas-api-openshift-ca" pod "5ff565e0-cada-4946-b535-f7cf03d8811e" (UID: "5ff565e0-cada-4946-b535-f7cf03d8811e"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 03 21:59:56.172529 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.172517 6524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82c4e70e-4765-48d3-80d3-8e0311abfda7-istio-podinfo\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:56.172577 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.172531 6524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hbh4z\" (UniqueName: \"kubernetes.io/projected/82c4e70e-4765-48d3-80d3-8e0311abfda7-kube-api-access-hbh4z\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:56.172577 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.172543 6524 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5ff565e0-cada-4946-b535-f7cf03d8811e-maas-api-openshift-ca\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:56.172577 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.172552 6524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82c4e70e-4765-48d3-80d3-8e0311abfda7-istio-token\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:56.172577 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.172562 6524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82c4e70e-4765-48d3-80d3-8e0311abfda7-istio-envoy\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:56.174137 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.174105 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ff565e0-cada-4946-b535-f7cf03d8811e-kube-api-access-rvjhx" pod "5ff565e0-cada-4946-b535-f7cf03d8811e" (UID: "5ff565e0-cada-4946-b535-f7cf03d8811e"). InnerVolumeSpecName "kube-api-access-rvjhx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 03 21:59:56.174233 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.174114 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ff565e0-cada-4946-b535-f7cf03d8811e-maas-api-xks-ca" pod "5ff565e0-cada-4946-b535-f7cf03d8811e" (UID: "5ff565e0-cada-4946-b535-f7cf03d8811e"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 03 21:59:56.274097 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.274052 6524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rvjhx\" (UniqueName: \"kubernetes.io/projected/5ff565e0-cada-4946-b535-f7cf03d8811e-kube-api-access-rvjhx\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:56.274097 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.274091 6524 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5ff565e0-cada-4946-b535-f7cf03d8811e-maas-api-xks-ca\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:56.357523 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.357494 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-8af3f3-7c9fcbb455-25425" Nov 03 21:59:56.430309 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.430281 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-8af3f3-655f44685b-wh9n2" Nov 03 21:59:56.478203 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.478155 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2e7d9548-49de-403f-8d4e-14eb05b8b338-kube-api-access-mqcb9\" (UniqueName: \"kubernetes.io/projected/2e7d9548-49de-403f-8d4e-14eb05b8b338-kube-api-access-mqcb9\") pod \"2e7d9548-49de-403f-8d4e-14eb05b8b338\" (UID: \"2e7d9548-49de-403f-8d4e-14eb05b8b338\") " Nov 03 21:59:56.478490 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.478281 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2e7d9548-49de-403f-8d4e-14eb05b8b338-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e7d9548-49de-403f-8d4e-14eb05b8b338-plugins-config-volume\") pod \"2e7d9548-49de-403f-8d4e-14eb05b8b338\" (UID: \"2e7d9548-49de-403f-8d4e-14eb05b8b338\") " Nov 03 21:59:56.479209 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.479111 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e7d9548-49de-403f-8d4e-14eb05b8b338-plugins-config-volume" pod "2e7d9548-49de-403f-8d4e-14eb05b8b338" (UID: "2e7d9548-49de-403f-8d4e-14eb05b8b338"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 03 21:59:56.482087 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.481974 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e7d9548-49de-403f-8d4e-14eb05b8b338-kube-api-access-mqcb9" pod "2e7d9548-49de-403f-8d4e-14eb05b8b338" (UID: "2e7d9548-49de-403f-8d4e-14eb05b8b338"). InnerVolumeSpecName "kube-api-access-mqcb9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 03 21:59:56.579854 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.579816 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/171f367e-e35d-4444-86b4-2842b49424fb-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/171f367e-e35d-4444-86b4-2842b49424fb-plugins-config-volume\") pod \"171f367e-e35d-4444-86b4-2842b49424fb\" (UID: \"171f367e-e35d-4444-86b4-2842b49424fb\") " Nov 03 21:59:56.580039 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.579902 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/171f367e-e35d-4444-86b4-2842b49424fb-kube-api-access-8xln6\" (UniqueName: \"kubernetes.io/projected/171f367e-e35d-4444-86b4-2842b49424fb-kube-api-access-8xln6\") pod \"171f367e-e35d-4444-86b4-2842b49424fb\" (UID: \"171f367e-e35d-4444-86b4-2842b49424fb\") " Nov 03 21:59:56.580274 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.580248 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/171f367e-e35d-4444-86b4-2842b49424fb-plugins-config-volume" pod "171f367e-e35d-4444-86b4-2842b49424fb" (UID: "171f367e-e35d-4444-86b4-2842b49424fb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 03 21:59:56.580334 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.580306 6524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mqcb9\" (UniqueName: \"kubernetes.io/projected/2e7d9548-49de-403f-8d4e-14eb05b8b338-kube-api-access-mqcb9\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:56.580334 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.580327 6524 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e7d9548-49de-403f-8d4e-14eb05b8b338-plugins-config-volume\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:56.582045 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.581981 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/171f367e-e35d-4444-86b4-2842b49424fb-kube-api-access-8xln6" pod "171f367e-e35d-4444-86b4-2842b49424fb" (UID: "171f367e-e35d-4444-86b4-2842b49424fb"). InnerVolumeSpecName "kube-api-access-8xln6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 03 21:59:56.684410 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.684369 6524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8xln6\" (UniqueName: \"kubernetes.io/projected/171f367e-e35d-4444-86b4-2842b49424fb-kube-api-access-8xln6\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:56.684410 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.684411 6524 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/171f367e-e35d-4444-86b4-2842b49424fb-plugins-config-volume\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:56.719177 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.719138 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9"] Nov 03 21:59:56.719641 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.719606 6524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" podUID="8462643c-89e1-4ee3-a413-44160da8e03d" containerName="istio-proxy" containerID="cri-o://3e1d409fe806d4bbaba0b8b39ab4c5c332583086ee3c8a99cb87a9fc7796744e" gracePeriod=30 Nov 03 21:59:56.823843 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.823801 6524 generic.go:356] "Generic (PLEG): container finished" podID="171f367e-e35d-4444-86b4-2842b49424fb" containerID="d977ae9bcb79749def02d4d045471895c08c7a6817720f02af879e2121d5970a" exitCode=0 Nov 03 21:59:56.824724 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.824688 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-8af3f3-655f44685b-wh9n2" event={"ID":"171f367e-e35d-4444-86b4-2842b49424fb","Type":"ContainerDied","Data":"d977ae9bcb79749def02d4d045471895c08c7a6817720f02af879e2121d5970a"} Nov 03 21:59:56.825005 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.824967 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-8af3f3-655f44685b-wh9n2" event={"ID":"171f367e-e35d-4444-86b4-2842b49424fb","Type":"ContainerDied","Data":"2ebde2cb2714b84fdc2a010a3bf3831a7361f503d6f6521552b2d723e6b5fb32"} Nov 03 21:59:56.825225 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.825199 6524 scope.go:122] "RemoveContainer" containerID="d977ae9bcb79749def02d4d045471895c08c7a6817720f02af879e2121d5970a" Nov 03 21:59:56.836555 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.825503 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-8af3f3-655f44685b-wh9n2" Nov 03 21:59:56.856151 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.855838 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-97163ab9-7t5ft" Nov 03 21:59:56.856556 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.856463 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-97163ab9-7t5ft" event={"ID":"5ff565e0-cada-4946-b535-f7cf03d8811e","Type":"ContainerDied","Data":"f3e0c69721a8949778f8a29865475bac217233ff97e874099d2f5c8c4da0eb50"} Nov 03 21:59:56.856556 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.856515 6524 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f3e0c69721a8949778f8a29865475bac217233ff97e874099d2f5c8c4da0eb50" Nov 03 21:59:56.859855 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.859740 6524 scope.go:122] "RemoveContainer" containerID="d977ae9bcb79749def02d4d045471895c08c7a6817720f02af879e2121d5970a" Nov 03 21:59:56.860891 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:59:56.860663 6524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d977ae9bcb79749def02d4d045471895c08c7a6817720f02af879e2121d5970a\": container with ID starting with d977ae9bcb79749def02d4d045471895c08c7a6817720f02af879e2121d5970a not found: ID does not exist" containerID="d977ae9bcb79749def02d4d045471895c08c7a6817720f02af879e2121d5970a" Nov 03 21:59:56.860891 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.860703 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d977ae9bcb79749def02d4d045471895c08c7a6817720f02af879e2121d5970a"} err="failed to get container status \"d977ae9bcb79749def02d4d045471895c08c7a6817720f02af879e2121d5970a\": rpc error: code = NotFound desc = could not find container \"d977ae9bcb79749def02d4d045471895c08c7a6817720f02af879e2121d5970a\": container with ID starting with d977ae9bcb79749def02d4d045471895c08c7a6817720f02af879e2121d5970a not found: ID does not exist" Nov 03 21:59:56.864817 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.863970 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" event={"ID":"82c4e70e-4765-48d3-80d3-8e0311abfda7","Type":"ContainerDied","Data":"89b0495e70faf4ac937912ee7e9b5689ff0094387706e96c0098c72e96baf9eb"} Nov 03 21:59:56.864817 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.864025 6524 scope.go:122] "RemoveContainer" containerID="a3cc8b56794c604d5e10892d7544628cc525e88f823336a6f11c3011e76cb3cc" Nov 03 21:59:56.864817 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.864185 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j" Nov 03 21:59:56.887196 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.884085 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-8af3f3-655f44685b-wh9n2"] Nov 03 21:59:56.887196 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.886081 6524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-8af3f3-655f44685b-wh9n2"] Nov 03 21:59:56.887949 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.887917 6524 generic.go:356] "Generic (PLEG): container finished" podID="2e7d9548-49de-403f-8d4e-14eb05b8b338" containerID="7dfa6ab0c2b15fb9851ea22605e8a6c45b019d97b5f25490e6834e84a74ea5b9" exitCode=0 Nov 03 21:59:56.888086 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.887982 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-8af3f3-7c9fcbb455-25425" Nov 03 21:59:56.888086 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.887977 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-8af3f3-7c9fcbb455-25425" event={"ID":"2e7d9548-49de-403f-8d4e-14eb05b8b338","Type":"ContainerDied","Data":"7dfa6ab0c2b15fb9851ea22605e8a6c45b019d97b5f25490e6834e84a74ea5b9"} Nov 03 21:59:56.888086 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.888044 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-8af3f3-7c9fcbb455-25425" event={"ID":"2e7d9548-49de-403f-8d4e-14eb05b8b338","Type":"ContainerDied","Data":"edbf33d8f2044e7e76dff44ad7749c582d4ce162604c2f8c692a8888d5bcfe34"} Nov 03 21:59:56.890367 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.890347 6524 scope.go:122] "RemoveContainer" containerID="7dfa6ab0c2b15fb9851ea22605e8a6c45b019d97b5f25490e6834e84a74ea5b9" Nov 03 21:59:56.899627 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.899599 6524 scope.go:122] "RemoveContainer" containerID="7dfa6ab0c2b15fb9851ea22605e8a6c45b019d97b5f25490e6834e84a74ea5b9" Nov 03 21:59:56.899968 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:59:56.899940 6524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7dfa6ab0c2b15fb9851ea22605e8a6c45b019d97b5f25490e6834e84a74ea5b9\": container with ID starting with 7dfa6ab0c2b15fb9851ea22605e8a6c45b019d97b5f25490e6834e84a74ea5b9 not found: ID does not exist" containerID="7dfa6ab0c2b15fb9851ea22605e8a6c45b019d97b5f25490e6834e84a74ea5b9" Nov 03 21:59:56.900031 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.899971 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7dfa6ab0c2b15fb9851ea22605e8a6c45b019d97b5f25490e6834e84a74ea5b9"} err="failed to get container status \"7dfa6ab0c2b15fb9851ea22605e8a6c45b019d97b5f25490e6834e84a74ea5b9\": rpc error: code = NotFound desc = could not find container \"7dfa6ab0c2b15fb9851ea22605e8a6c45b019d97b5f25490e6834e84a74ea5b9\": container with ID starting with 7dfa6ab0c2b15fb9851ea22605e8a6c45b019d97b5f25490e6834e84a74ea5b9 not found: ID does not exist" Nov 03 21:59:56.905914 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.905887 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j"] Nov 03 21:59:56.910064 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.910035 6524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-c0caef-openshift-default-58fbf8db4f-2627j"] Nov 03 21:59:56.925944 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.925919 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-8af3f3-7c9fcbb455-25425"] Nov 03 21:59:56.931249 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:56.931218 6524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-8af3f3-7c9fcbb455-25425"] Nov 03 21:59:57.333060 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:57.333035 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" Nov 03 21:59:57.492831 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:57.492746 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7a83deb0-69e5-4c9d-a573-235be3ff17ef-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a83deb0-69e5-4c9d-a573-235be3ff17ef-istio-envoy\") pod \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\" (UID: \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\") " Nov 03 21:59:57.492831 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:57.492793 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7a83deb0-69e5-4c9d-a573-235be3ff17ef-kube-api-access-zqr48\" (UniqueName: \"kubernetes.io/projected/7a83deb0-69e5-4c9d-a573-235be3ff17ef-kube-api-access-zqr48\") pod \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\" (UID: \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\") " Nov 03 21:59:57.493042 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:57.492854 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7a83deb0-69e5-4c9d-a573-235be3ff17ef-istio-token\" (UniqueName: \"kubernetes.io/projected/7a83deb0-69e5-4c9d-a573-235be3ff17ef-istio-token\") pod \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\" (UID: \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\") " Nov 03 21:59:57.493042 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:57.492882 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7a83deb0-69e5-4c9d-a573-235be3ff17ef-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a83deb0-69e5-4c9d-a573-235be3ff17ef-istiod-ca-cert\") pod \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\" (UID: \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\") " Nov 03 21:59:57.493042 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:57.492924 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7a83deb0-69e5-4c9d-a573-235be3ff17ef-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a83deb0-69e5-4c9d-a573-235be3ff17ef-workload-certs\") pod \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\" (UID: \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\") " Nov 03 21:59:57.493042 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:57.492977 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7a83deb0-69e5-4c9d-a573-235be3ff17ef-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a83deb0-69e5-4c9d-a573-235be3ff17ef-istio-data\") pod \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\" (UID: \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\") " Nov 03 21:59:57.493042 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:57.493012 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7a83deb0-69e5-4c9d-a573-235be3ff17ef-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a83deb0-69e5-4c9d-a573-235be3ff17ef-workload-socket\") pod \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\" (UID: \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\") " Nov 03 21:59:57.493338 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:57.493063 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7a83deb0-69e5-4c9d-a573-235be3ff17ef-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a83deb0-69e5-4c9d-a573-235be3ff17ef-istio-podinfo\") pod \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\" (UID: \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\") " Nov 03 21:59:57.493338 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:57.493111 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7a83deb0-69e5-4c9d-a573-235be3ff17ef-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a83deb0-69e5-4c9d-a573-235be3ff17ef-credential-socket\") pod \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\" (UID: \"7a83deb0-69e5-4c9d-a573-235be3ff17ef\") " Nov 03 21:59:57.493437 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:57.493306 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a83deb0-69e5-4c9d-a573-235be3ff17ef-workload-certs" pod "7a83deb0-69e5-4c9d-a573-235be3ff17ef" (UID: "7a83deb0-69e5-4c9d-a573-235be3ff17ef"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 21:59:57.493437 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:57.493369 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a83deb0-69e5-4c9d-a573-235be3ff17ef-workload-socket" pod "7a83deb0-69e5-4c9d-a573-235be3ff17ef" (UID: "7a83deb0-69e5-4c9d-a573-235be3ff17ef"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 21:59:57.493437 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:57.493412 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a83deb0-69e5-4c9d-a573-235be3ff17ef-istiod-ca-cert" pod "7a83deb0-69e5-4c9d-a573-235be3ff17ef" (UID: "7a83deb0-69e5-4c9d-a573-235be3ff17ef"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 03 21:59:57.493566 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:57.493545 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a83deb0-69e5-4c9d-a573-235be3ff17ef-credential-socket" pod "7a83deb0-69e5-4c9d-a573-235be3ff17ef" (UID: "7a83deb0-69e5-4c9d-a573-235be3ff17ef"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 21:59:57.493631 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:57.493609 6524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a83deb0-69e5-4c9d-a573-235be3ff17ef-istiod-ca-cert\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:57.493681 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:57.493634 6524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a83deb0-69e5-4c9d-a573-235be3ff17ef-workload-certs\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:57.493681 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:57.493650 6524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a83deb0-69e5-4c9d-a573-235be3ff17ef-workload-socket\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:57.493794 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:57.493761 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a83deb0-69e5-4c9d-a573-235be3ff17ef-istio-data" pod "7a83deb0-69e5-4c9d-a573-235be3ff17ef" (UID: "7a83deb0-69e5-4c9d-a573-235be3ff17ef"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 21:59:57.495046 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:57.495023 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a83deb0-69e5-4c9d-a573-235be3ff17ef-istio-token" pod "7a83deb0-69e5-4c9d-a573-235be3ff17ef" (UID: "7a83deb0-69e5-4c9d-a573-235be3ff17ef"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 03 21:59:57.495302 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:57.495286 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7a83deb0-69e5-4c9d-a573-235be3ff17ef-istio-podinfo" pod "7a83deb0-69e5-4c9d-a573-235be3ff17ef" (UID: "7a83deb0-69e5-4c9d-a573-235be3ff17ef"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 03 21:59:57.495453 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:57.495436 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a83deb0-69e5-4c9d-a573-235be3ff17ef-kube-api-access-zqr48" pod "7a83deb0-69e5-4c9d-a573-235be3ff17ef" (UID: "7a83deb0-69e5-4c9d-a573-235be3ff17ef"). InnerVolumeSpecName "kube-api-access-zqr48". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 03 21:59:57.495453 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:57.495442 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a83deb0-69e5-4c9d-a573-235be3ff17ef-istio-envoy" pod "7a83deb0-69e5-4c9d-a573-235be3ff17ef" (UID: "7a83deb0-69e5-4c9d-a573-235be3ff17ef"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 21:59:57.595323 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:57.595287 6524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a83deb0-69e5-4c9d-a573-235be3ff17ef-istio-envoy\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:57.595323 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:57.595326 6524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zqr48\" (UniqueName: \"kubernetes.io/projected/7a83deb0-69e5-4c9d-a573-235be3ff17ef-kube-api-access-zqr48\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:57.595541 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:57.595341 6524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a83deb0-69e5-4c9d-a573-235be3ff17ef-istio-token\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:57.595541 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:57.595354 6524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a83deb0-69e5-4c9d-a573-235be3ff17ef-istio-data\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:57.595541 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:57.595369 6524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a83deb0-69e5-4c9d-a573-235be3ff17ef-istio-podinfo\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:57.595541 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:57.595382 6524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a83deb0-69e5-4c9d-a573-235be3ff17ef-credential-socket\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:57.897824 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:57.897789 6524 generic.go:356] "Generic (PLEG): container finished" podID="7a83deb0-69e5-4c9d-a573-235be3ff17ef" containerID="640fe0bf5b0c90c23ecf2082960fe45c8714919d32574ca3ee7751064084e229" exitCode=0 Nov 03 21:59:57.898329 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:57.897854 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" Nov 03 21:59:57.898329 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:57.897865 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" event={"ID":"7a83deb0-69e5-4c9d-a573-235be3ff17ef","Type":"ContainerDied","Data":"640fe0bf5b0c90c23ecf2082960fe45c8714919d32574ca3ee7751064084e229"} Nov 03 21:59:57.898329 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:57.897920 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2" event={"ID":"7a83deb0-69e5-4c9d-a573-235be3ff17ef","Type":"ContainerDied","Data":"00982e4bee9dbb6e97b337ed0b094aeeba940fda5aeff1b1d35934aed4d5c0db"} Nov 03 21:59:57.898329 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:57.897941 6524 scope.go:122] "RemoveContainer" containerID="640fe0bf5b0c90c23ecf2082960fe45c8714919d32574ca3ee7751064084e229" Nov 03 21:59:57.908603 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:57.908581 6524 scope.go:122] "RemoveContainer" containerID="640fe0bf5b0c90c23ecf2082960fe45c8714919d32574ca3ee7751064084e229" Nov 03 21:59:57.908961 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:59:57.908906 6524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"640fe0bf5b0c90c23ecf2082960fe45c8714919d32574ca3ee7751064084e229\": container with ID starting with 640fe0bf5b0c90c23ecf2082960fe45c8714919d32574ca3ee7751064084e229 not found: ID does not exist" containerID="640fe0bf5b0c90c23ecf2082960fe45c8714919d32574ca3ee7751064084e229" Nov 03 21:59:57.909074 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:57.908970 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"640fe0bf5b0c90c23ecf2082960fe45c8714919d32574ca3ee7751064084e229"} err="failed to get container status \"640fe0bf5b0c90c23ecf2082960fe45c8714919d32574ca3ee7751064084e229\": rpc error: code = NotFound desc = could not find container \"640fe0bf5b0c90c23ecf2082960fe45c8714919d32574ca3ee7751064084e229\": container with ID starting with 640fe0bf5b0c90c23ecf2082960fe45c8714919d32574ca3ee7751064084e229 not found: ID does not exist" Nov 03 21:59:57.924025 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:57.923998 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2"] Nov 03 21:59:57.935052 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:57.935022 6524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-85b4ff-openshift-default-84c9ccb77-g8hx2"] Nov 03 21:59:58.440091 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:58.440061 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" Nov 03 21:59:58.604265 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:58.604227 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/efcc7e46-86ce-427c-a5d7-99878a682ac1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efcc7e46-86ce-427c-a5d7-99878a682ac1-istio-data\") pod \"efcc7e46-86ce-427c-a5d7-99878a682ac1\" (UID: \"efcc7e46-86ce-427c-a5d7-99878a682ac1\") " Nov 03 21:59:58.604265 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:58.604272 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/efcc7e46-86ce-427c-a5d7-99878a682ac1-kube-api-access-hvzdg\" (UniqueName: \"kubernetes.io/projected/efcc7e46-86ce-427c-a5d7-99878a682ac1-kube-api-access-hvzdg\") pod \"efcc7e46-86ce-427c-a5d7-99878a682ac1\" (UID: \"efcc7e46-86ce-427c-a5d7-99878a682ac1\") " Nov 03 21:59:58.604605 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:58.604334 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/efcc7e46-86ce-427c-a5d7-99878a682ac1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efcc7e46-86ce-427c-a5d7-99878a682ac1-istio-envoy\") pod \"efcc7e46-86ce-427c-a5d7-99878a682ac1\" (UID: \"efcc7e46-86ce-427c-a5d7-99878a682ac1\") " Nov 03 21:59:58.604605 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:58.604359 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/efcc7e46-86ce-427c-a5d7-99878a682ac1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efcc7e46-86ce-427c-a5d7-99878a682ac1-workload-socket\") pod \"efcc7e46-86ce-427c-a5d7-99878a682ac1\" (UID: \"efcc7e46-86ce-427c-a5d7-99878a682ac1\") " Nov 03 21:59:58.604605 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:58.604401 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/efcc7e46-86ce-427c-a5d7-99878a682ac1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efcc7e46-86ce-427c-a5d7-99878a682ac1-workload-certs\") pod \"efcc7e46-86ce-427c-a5d7-99878a682ac1\" (UID: \"efcc7e46-86ce-427c-a5d7-99878a682ac1\") " Nov 03 21:59:58.604605 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:58.604426 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/efcc7e46-86ce-427c-a5d7-99878a682ac1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efcc7e46-86ce-427c-a5d7-99878a682ac1-istio-podinfo\") pod \"efcc7e46-86ce-427c-a5d7-99878a682ac1\" (UID: \"efcc7e46-86ce-427c-a5d7-99878a682ac1\") " Nov 03 21:59:58.604605 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:58.604465 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/efcc7e46-86ce-427c-a5d7-99878a682ac1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efcc7e46-86ce-427c-a5d7-99878a682ac1-credential-socket\") pod \"efcc7e46-86ce-427c-a5d7-99878a682ac1\" (UID: \"efcc7e46-86ce-427c-a5d7-99878a682ac1\") " Nov 03 21:59:58.604605 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:58.604536 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/efcc7e46-86ce-427c-a5d7-99878a682ac1-istio-token\" (UniqueName: \"kubernetes.io/projected/efcc7e46-86ce-427c-a5d7-99878a682ac1-istio-token\") pod \"efcc7e46-86ce-427c-a5d7-99878a682ac1\" (UID: \"efcc7e46-86ce-427c-a5d7-99878a682ac1\") " Nov 03 21:59:58.604605 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:58.604570 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/efcc7e46-86ce-427c-a5d7-99878a682ac1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efcc7e46-86ce-427c-a5d7-99878a682ac1-istiod-ca-cert\") pod \"efcc7e46-86ce-427c-a5d7-99878a682ac1\" (UID: \"efcc7e46-86ce-427c-a5d7-99878a682ac1\") " Nov 03 21:59:58.604605 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:58.604582 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efcc7e46-86ce-427c-a5d7-99878a682ac1-istio-data" pod "efcc7e46-86ce-427c-a5d7-99878a682ac1" (UID: "efcc7e46-86ce-427c-a5d7-99878a682ac1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 21:59:58.604968 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:58.604695 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efcc7e46-86ce-427c-a5d7-99878a682ac1-workload-certs" pod "efcc7e46-86ce-427c-a5d7-99878a682ac1" (UID: "efcc7e46-86ce-427c-a5d7-99878a682ac1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 21:59:58.604968 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:58.604708 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efcc7e46-86ce-427c-a5d7-99878a682ac1-workload-socket" pod "efcc7e46-86ce-427c-a5d7-99878a682ac1" (UID: "efcc7e46-86ce-427c-a5d7-99878a682ac1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 21:59:58.604968 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:58.604798 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efcc7e46-86ce-427c-a5d7-99878a682ac1-credential-socket" pod "efcc7e46-86ce-427c-a5d7-99878a682ac1" (UID: "efcc7e46-86ce-427c-a5d7-99878a682ac1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 21:59:58.605115 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:58.604979 6524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efcc7e46-86ce-427c-a5d7-99878a682ac1-istio-data\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:58.605115 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:58.605001 6524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efcc7e46-86ce-427c-a5d7-99878a682ac1-workload-socket\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:58.605115 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:58.605015 6524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efcc7e46-86ce-427c-a5d7-99878a682ac1-workload-certs\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:58.605115 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:58.605015 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efcc7e46-86ce-427c-a5d7-99878a682ac1-istiod-ca-cert" pod "efcc7e46-86ce-427c-a5d7-99878a682ac1" (UID: "efcc7e46-86ce-427c-a5d7-99878a682ac1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 03 21:59:58.605115 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:58.605025 6524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efcc7e46-86ce-427c-a5d7-99878a682ac1-credential-socket\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:58.606716 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:58.606687 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efcc7e46-86ce-427c-a5d7-99878a682ac1-istio-token" pod "efcc7e46-86ce-427c-a5d7-99878a682ac1" (UID: "efcc7e46-86ce-427c-a5d7-99878a682ac1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 03 21:59:58.606815 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:58.606740 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/efcc7e46-86ce-427c-a5d7-99878a682ac1-istio-podinfo" pod "efcc7e46-86ce-427c-a5d7-99878a682ac1" (UID: "efcc7e46-86ce-427c-a5d7-99878a682ac1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 03 21:59:58.606976 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:58.606957 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efcc7e46-86ce-427c-a5d7-99878a682ac1-istio-envoy" pod "efcc7e46-86ce-427c-a5d7-99878a682ac1" (UID: "efcc7e46-86ce-427c-a5d7-99878a682ac1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 21:59:58.607040 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:58.606981 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efcc7e46-86ce-427c-a5d7-99878a682ac1-kube-api-access-hvzdg" pod "efcc7e46-86ce-427c-a5d7-99878a682ac1" (UID: "efcc7e46-86ce-427c-a5d7-99878a682ac1"). InnerVolumeSpecName "kube-api-access-hvzdg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 03 21:59:58.705576 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:58.705539 6524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efcc7e46-86ce-427c-a5d7-99878a682ac1-istio-envoy\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:58.705576 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:58.705571 6524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efcc7e46-86ce-427c-a5d7-99878a682ac1-istio-podinfo\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:58.705576 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:58.705584 6524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efcc7e46-86ce-427c-a5d7-99878a682ac1-istio-token\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:58.705804 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:58.705597 6524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efcc7e46-86ce-427c-a5d7-99878a682ac1-istiod-ca-cert\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:58.705804 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:58.705610 6524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hvzdg\" (UniqueName: \"kubernetes.io/projected/efcc7e46-86ce-427c-a5d7-99878a682ac1-kube-api-access-hvzdg\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 21:59:58.767956 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:58.767916 6524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="171f367e-e35d-4444-86b4-2842b49424fb" path="/var/lib/kubelet/pods/171f367e-e35d-4444-86b4-2842b49424fb/volumes" Nov 03 21:59:58.768384 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:58.768367 6524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2e7d9548-49de-403f-8d4e-14eb05b8b338" path="/var/lib/kubelet/pods/2e7d9548-49de-403f-8d4e-14eb05b8b338/volumes" Nov 03 21:59:58.768779 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:58.768767 6524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7a83deb0-69e5-4c9d-a573-235be3ff17ef" path="/var/lib/kubelet/pods/7a83deb0-69e5-4c9d-a573-235be3ff17ef/volumes" Nov 03 21:59:58.769457 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:58.769442 6524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="82c4e70e-4765-48d3-80d3-8e0311abfda7" path="/var/lib/kubelet/pods/82c4e70e-4765-48d3-80d3-8e0311abfda7/volumes" Nov 03 21:59:58.906394 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:58.906302 6524 generic.go:356] "Generic (PLEG): container finished" podID="efcc7e46-86ce-427c-a5d7-99878a682ac1" containerID="f00351cebd65d645d82b2ea0a3592d9e03a611ebfe205b7c9edffc54c124aaec" exitCode=0 Nov 03 21:59:58.906394 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:58.906353 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" Nov 03 21:59:58.906880 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:58.906390 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" event={"ID":"efcc7e46-86ce-427c-a5d7-99878a682ac1","Type":"ContainerDied","Data":"f00351cebd65d645d82b2ea0a3592d9e03a611ebfe205b7c9edffc54c124aaec"} Nov 03 21:59:58.906880 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:58.906449 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt" event={"ID":"efcc7e46-86ce-427c-a5d7-99878a682ac1","Type":"ContainerDied","Data":"3df408bef998235765aeb3e9e4f1953244b38be3bf851d417210bfd186ce5454"} Nov 03 21:59:58.906880 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:58.906477 6524 scope.go:122] "RemoveContainer" containerID="f00351cebd65d645d82b2ea0a3592d9e03a611ebfe205b7c9edffc54c124aaec" Nov 03 21:59:58.915601 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:58.915579 6524 scope.go:122] "RemoveContainer" containerID="f00351cebd65d645d82b2ea0a3592d9e03a611ebfe205b7c9edffc54c124aaec" Nov 03 21:59:58.915910 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:59:58.915890 6524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f00351cebd65d645d82b2ea0a3592d9e03a611ebfe205b7c9edffc54c124aaec\": container with ID starting with f00351cebd65d645d82b2ea0a3592d9e03a611ebfe205b7c9edffc54c124aaec not found: ID does not exist" containerID="f00351cebd65d645d82b2ea0a3592d9e03a611ebfe205b7c9edffc54c124aaec" Nov 03 21:59:58.915969 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:58.915918 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f00351cebd65d645d82b2ea0a3592d9e03a611ebfe205b7c9edffc54c124aaec"} err="failed to get container status \"f00351cebd65d645d82b2ea0a3592d9e03a611ebfe205b7c9edffc54c124aaec\": rpc error: code = NotFound desc = could not find container \"f00351cebd65d645d82b2ea0a3592d9e03a611ebfe205b7c9edffc54c124aaec\": container with ID starting with f00351cebd65d645d82b2ea0a3592d9e03a611ebfe205b7c9edffc54c124aaec not found: ID does not exist" Nov 03 21:59:58.924974 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:58.924949 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt"] Nov 03 21:59:58.931135 ip-10-0-137-7 kubenswrapper[6524]: I1103 21:59:58.931089 6524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-f4fb1b-openshift-default-78b8dffc77-8p4rt"] Nov 03 21:59:59.707583 ip-10-0-137-7 kubenswrapper[6524]: E1103 21:59:59.707539 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 22:00:00.099241 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.099202 6524 patch_prober.go:29] interesting pod/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Nov 03 22:00:00.099621 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.099270 6524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" podUID="8462643c-89e1-4ee3-a413-44160da8e03d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Nov 03 22:00:00.157086 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.157052 6524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-fc16de-29895720-kgtn8"] Nov 03 22:00:00.157457 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.157441 6524 cpu_manager.go:414] "containerMap: removing container" podUID="82c4e70e-4765-48d3-80d3-8e0311abfda7" containerName="istio-proxy" Nov 03 22:00:00.157503 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.157458 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="82c4e70e-4765-48d3-80d3-8e0311abfda7" containerName="istio-proxy" Nov 03 22:00:00.157503 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.157470 6524 cpu_manager.go:414] "containerMap: removing container" podUID="5ff565e0-cada-4946-b535-f7cf03d8811e" containerName="revoke-keys" Nov 03 22:00:00.157503 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.157476 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5ff565e0-cada-4946-b535-f7cf03d8811e" containerName="revoke-keys" Nov 03 22:00:00.157503 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.157487 6524 cpu_manager.go:414] "containerMap: removing container" podUID="7a83deb0-69e5-4c9d-a573-235be3ff17ef" containerName="istio-proxy" Nov 03 22:00:00.157503 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.157492 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7a83deb0-69e5-4c9d-a573-235be3ff17ef" containerName="istio-proxy" Nov 03 22:00:00.157652 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.157506 6524 cpu_manager.go:414] "containerMap: removing container" podUID="efcc7e46-86ce-427c-a5d7-99878a682ac1" containerName="istio-proxy" Nov 03 22:00:00.157652 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.157511 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="efcc7e46-86ce-427c-a5d7-99878a682ac1" containerName="istio-proxy" Nov 03 22:00:00.157652 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.157532 6524 cpu_manager.go:414] "containerMap: removing container" podUID="2e7d9548-49de-403f-8d4e-14eb05b8b338" containerName="payload-pre-processing" Nov 03 22:00:00.157652 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.157537 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2e7d9548-49de-403f-8d4e-14eb05b8b338" containerName="payload-pre-processing" Nov 03 22:00:00.157652 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.157543 6524 cpu_manager.go:414] "containerMap: removing container" podUID="171f367e-e35d-4444-86b4-2842b49424fb" containerName="payload-processing" Nov 03 22:00:00.157652 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.157548 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="171f367e-e35d-4444-86b4-2842b49424fb" containerName="payload-processing" Nov 03 22:00:00.157652 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.157599 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="2e7d9548-49de-403f-8d4e-14eb05b8b338" containerName="payload-pre-processing" Nov 03 22:00:00.157652 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.157606 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="171f367e-e35d-4444-86b4-2842b49424fb" containerName="payload-processing" Nov 03 22:00:00.157652 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.157612 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="82c4e70e-4765-48d3-80d3-8e0311abfda7" containerName="istio-proxy" Nov 03 22:00:00.157652 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.157618 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="82c4e70e-4765-48d3-80d3-8e0311abfda7" containerName="istio-proxy" Nov 03 22:00:00.157652 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.157624 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="5ff565e0-cada-4946-b535-f7cf03d8811e" containerName="revoke-keys" Nov 03 22:00:00.157652 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.157630 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="7a83deb0-69e5-4c9d-a573-235be3ff17ef" containerName="istio-proxy" Nov 03 22:00:00.157652 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.157636 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="efcc7e46-86ce-427c-a5d7-99878a682ac1" containerName="istio-proxy" Nov 03 22:00:00.162277 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.162259 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-fc16de-29895720-kgtn8" Nov 03 22:00:00.167572 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.167537 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5n7j7\"" Nov 03 22:00:00.170547 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.170523 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-fc16de-29895720-kgtn8"] Nov 03 22:00:00.219076 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.219030 6524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv" podUID="851596d1-02e4-42b9-bdab-19a9321a2a04" containerName="main" containerID="cri-o://c582bc14a1410dc15a1fe39319b6e029db8853db184a89bb736565e45f67eaf6" gracePeriod=45 Nov 03 22:00:00.223481 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.223452 6524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv" podUID="851596d1-02e4-42b9-bdab-19a9321a2a04" containerName="main" probeResult="failure" output="Get \"https://10.133.0.31:8000/ready\": EOF" Nov 03 22:00:00.320941 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.320899 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/587e7995-5cb6-4f61-9f3e-a3fe113a39ad-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-fc16de-29895720-kgtn8\" (UID: \"587e7995-5cb6-4f61-9f3e-a3fe113a39ad\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-fc16de-29895720-kgtn8" Nov 03 22:00:00.321162 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.321024 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhjnm\" (UniqueName: \"kubernetes.io/projected/587e7995-5cb6-4f61-9f3e-a3fe113a39ad-kube-api-access-hhjnm\") pod \"maas-api-key-cleanup-e2e-worker-w4-fc16de-29895720-kgtn8\" (UID: \"587e7995-5cb6-4f61-9f3e-a3fe113a39ad\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-fc16de-29895720-kgtn8" Nov 03 22:00:00.421986 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.421940 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/587e7995-5cb6-4f61-9f3e-a3fe113a39ad-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-fc16de-29895720-kgtn8\" (UID: \"587e7995-5cb6-4f61-9f3e-a3fe113a39ad\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-fc16de-29895720-kgtn8" Nov 03 22:00:00.422169 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.422030 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhjnm\" (UniqueName: \"kubernetes.io/projected/587e7995-5cb6-4f61-9f3e-a3fe113a39ad-kube-api-access-hhjnm\") pod \"maas-api-key-cleanup-e2e-worker-w4-fc16de-29895720-kgtn8\" (UID: \"587e7995-5cb6-4f61-9f3e-a3fe113a39ad\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-fc16de-29895720-kgtn8" Nov 03 22:00:00.422673 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.422653 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/587e7995-5cb6-4f61-9f3e-a3fe113a39ad-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-fc16de-29895720-kgtn8\" (UID: \"587e7995-5cb6-4f61-9f3e-a3fe113a39ad\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-fc16de-29895720-kgtn8" Nov 03 22:00:00.434289 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.434256 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhjnm\" (UniqueName: \"kubernetes.io/projected/587e7995-5cb6-4f61-9f3e-a3fe113a39ad-kube-api-access-hhjnm\") pod \"maas-api-key-cleanup-e2e-worker-w4-fc16de-29895720-kgtn8\" (UID: \"587e7995-5cb6-4f61-9f3e-a3fe113a39ad\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-fc16de-29895720-kgtn8" Nov 03 22:00:00.482099 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.476661 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-fc16de-29895720-kgtn8" Nov 03 22:00:00.498196 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.498166 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv" Nov 03 22:00:00.625926 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.625891 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/851596d1-02e4-42b9-bdab-19a9321a2a04-dshm\" (UniqueName: \"kubernetes.io/empty-dir/851596d1-02e4-42b9-bdab-19a9321a2a04-dshm\") pod \"851596d1-02e4-42b9-bdab-19a9321a2a04\" (UID: \"851596d1-02e4-42b9-bdab-19a9321a2a04\") " Nov 03 22:00:00.626114 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.625953 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/851596d1-02e4-42b9-bdab-19a9321a2a04-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/851596d1-02e4-42b9-bdab-19a9321a2a04-tmp-dir\") pod \"851596d1-02e4-42b9-bdab-19a9321a2a04\" (UID: \"851596d1-02e4-42b9-bdab-19a9321a2a04\") " Nov 03 22:00:00.626114 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.625993 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/851596d1-02e4-42b9-bdab-19a9321a2a04-tls-certs\" (UniqueName: \"kubernetes.io/secret/851596d1-02e4-42b9-bdab-19a9321a2a04-tls-certs\") pod \"851596d1-02e4-42b9-bdab-19a9321a2a04\" (UID: \"851596d1-02e4-42b9-bdab-19a9321a2a04\") " Nov 03 22:00:00.626114 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.626030 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/851596d1-02e4-42b9-bdab-19a9321a2a04-home\" (UniqueName: \"kubernetes.io/empty-dir/851596d1-02e4-42b9-bdab-19a9321a2a04-home\") pod \"851596d1-02e4-42b9-bdab-19a9321a2a04\" (UID: \"851596d1-02e4-42b9-bdab-19a9321a2a04\") " Nov 03 22:00:00.626114 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.626068 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/851596d1-02e4-42b9-bdab-19a9321a2a04-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/851596d1-02e4-42b9-bdab-19a9321a2a04-model-cache\") pod \"851596d1-02e4-42b9-bdab-19a9321a2a04\" (UID: \"851596d1-02e4-42b9-bdab-19a9321a2a04\") " Nov 03 22:00:00.626383 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.626186 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/851596d1-02e4-42b9-bdab-19a9321a2a04-kube-api-access-96fjc\" (UniqueName: \"kubernetes.io/projected/851596d1-02e4-42b9-bdab-19a9321a2a04-kube-api-access-96fjc\") pod \"851596d1-02e4-42b9-bdab-19a9321a2a04\" (UID: \"851596d1-02e4-42b9-bdab-19a9321a2a04\") " Nov 03 22:00:00.626704 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.626649 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/851596d1-02e4-42b9-bdab-19a9321a2a04-model-cache" pod "851596d1-02e4-42b9-bdab-19a9321a2a04" (UID: "851596d1-02e4-42b9-bdab-19a9321a2a04"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 22:00:00.626704 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.626659 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/851596d1-02e4-42b9-bdab-19a9321a2a04-home" pod "851596d1-02e4-42b9-bdab-19a9321a2a04" (UID: "851596d1-02e4-42b9-bdab-19a9321a2a04"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 22:00:00.626855 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.626702 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/851596d1-02e4-42b9-bdab-19a9321a2a04-tmp-dir" pod "851596d1-02e4-42b9-bdab-19a9321a2a04" (UID: "851596d1-02e4-42b9-bdab-19a9321a2a04"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 22:00:00.629219 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.629186 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/851596d1-02e4-42b9-bdab-19a9321a2a04-tls-certs" pod "851596d1-02e4-42b9-bdab-19a9321a2a04" (UID: "851596d1-02e4-42b9-bdab-19a9321a2a04"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 03 22:00:00.629219 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.629210 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/851596d1-02e4-42b9-bdab-19a9321a2a04-kube-api-access-96fjc" pod "851596d1-02e4-42b9-bdab-19a9321a2a04" (UID: "851596d1-02e4-42b9-bdab-19a9321a2a04"). InnerVolumeSpecName "kube-api-access-96fjc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 03 22:00:00.629328 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.629227 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/851596d1-02e4-42b9-bdab-19a9321a2a04-dshm" pod "851596d1-02e4-42b9-bdab-19a9321a2a04" (UID: "851596d1-02e4-42b9-bdab-19a9321a2a04"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 22:00:00.727607 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.727574 6524 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/851596d1-02e4-42b9-bdab-19a9321a2a04-dshm\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:00.727607 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.727606 6524 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/851596d1-02e4-42b9-bdab-19a9321a2a04-tmp-dir\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:00.727787 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.727617 6524 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/851596d1-02e4-42b9-bdab-19a9321a2a04-tls-certs\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:00.727787 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.727628 6524 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/851596d1-02e4-42b9-bdab-19a9321a2a04-home\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:00.727787 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.727640 6524 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/851596d1-02e4-42b9-bdab-19a9321a2a04-model-cache\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:00.727787 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.727653 6524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96fjc\" (UniqueName: \"kubernetes.io/projected/851596d1-02e4-42b9-bdab-19a9321a2a04-kube-api-access-96fjc\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:00.769344 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.769308 6524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="efcc7e46-86ce-427c-a5d7-99878a682ac1" path="/var/lib/kubelet/pods/efcc7e46-86ce-427c-a5d7-99878a682ac1/volumes" Nov 03 22:00:00.778591 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.778564 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-fc16de-29895720-kgtn8"] Nov 03 22:00:00.781481 ip-10-0-137-7 kubenswrapper[6524]: W1103 22:00:00.781454 6524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod587e7995_5cb6_4f61_9f3e_a3fe113a39ad.slice/crio-6df65979af349819dc12ff2563675a7348ee6645348a44afa56fbe8fe1ec6ed1 WatchSource:0}: Error finding container 6df65979af349819dc12ff2563675a7348ee6645348a44afa56fbe8fe1ec6ed1: Status 404 returned error can't find the container with id 6df65979af349819dc12ff2563675a7348ee6645348a44afa56fbe8fe1ec6ed1 Nov 03 22:00:00.885531 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.885503 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" Nov 03 22:00:00.919945 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.919865 6524 generic.go:356] "Generic (PLEG): container finished" podID="851596d1-02e4-42b9-bdab-19a9321a2a04" containerID="c582bc14a1410dc15a1fe39319b6e029db8853db184a89bb736565e45f67eaf6" exitCode=0 Nov 03 22:00:00.919945 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.919906 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv" Nov 03 22:00:00.920192 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.919943 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv" event={"ID":"851596d1-02e4-42b9-bdab-19a9321a2a04","Type":"ContainerDied","Data":"c582bc14a1410dc15a1fe39319b6e029db8853db184a89bb736565e45f67eaf6"} Nov 03 22:00:00.920192 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.919996 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv" event={"ID":"851596d1-02e4-42b9-bdab-19a9321a2a04","Type":"ContainerDied","Data":"8f180beb1f86ab2301fc421a361b40920344a47f053dfe5e138bfb823649cb1d"} Nov 03 22:00:00.920192 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.920019 6524 scope.go:122] "RemoveContainer" containerID="c582bc14a1410dc15a1fe39319b6e029db8853db184a89bb736565e45f67eaf6" Nov 03 22:00:00.921834 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.921808 6524 generic.go:356] "Generic (PLEG): container finished" podID="779feacc-19e1-4b9e-9968-896c178c78f8" containerID="4a48814aeb26f05ca436d6300a258f42e728939c47bb85f7fbb0ae1fafa98bce" exitCode=0 Nov 03 22:00:00.921954 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.921858 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" Nov 03 22:00:00.921954 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.921858 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" event={"ID":"779feacc-19e1-4b9e-9968-896c178c78f8","Type":"ContainerDied","Data":"4a48814aeb26f05ca436d6300a258f42e728939c47bb85f7fbb0ae1fafa98bce"} Nov 03 22:00:00.922115 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.921958 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9" event={"ID":"779feacc-19e1-4b9e-9968-896c178c78f8","Type":"ContainerDied","Data":"1eac504f6610f08b657baab298c3fbea604b050a7d7ce872570634afabb25c63"} Nov 03 22:00:00.923519 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.923495 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-fc16de-29895720-kgtn8" event={"ID":"587e7995-5cb6-4f61-9f3e-a3fe113a39ad","Type":"ContainerStarted","Data":"6df65979af349819dc12ff2563675a7348ee6645348a44afa56fbe8fe1ec6ed1"} Nov 03 22:00:00.932927 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.932853 6524 scope.go:122] "RemoveContainer" containerID="c582bc14a1410dc15a1fe39319b6e029db8853db184a89bb736565e45f67eaf6" Nov 03 22:00:00.933261 ip-10-0-137-7 kubenswrapper[6524]: E1103 22:00:00.933237 6524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c582bc14a1410dc15a1fe39319b6e029db8853db184a89bb736565e45f67eaf6\": container with ID starting with c582bc14a1410dc15a1fe39319b6e029db8853db184a89bb736565e45f67eaf6 not found: ID does not exist" containerID="c582bc14a1410dc15a1fe39319b6e029db8853db184a89bb736565e45f67eaf6" Nov 03 22:00:00.933438 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.933271 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c582bc14a1410dc15a1fe39319b6e029db8853db184a89bb736565e45f67eaf6"} err="failed to get container status \"c582bc14a1410dc15a1fe39319b6e029db8853db184a89bb736565e45f67eaf6\": rpc error: code = NotFound desc = could not find container \"c582bc14a1410dc15a1fe39319b6e029db8853db184a89bb736565e45f67eaf6\": container with ID starting with c582bc14a1410dc15a1fe39319b6e029db8853db184a89bb736565e45f67eaf6 not found: ID does not exist" Nov 03 22:00:00.933438 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.933293 6524 scope.go:122] "RemoveContainer" containerID="4a48814aeb26f05ca436d6300a258f42e728939c47bb85f7fbb0ae1fafa98bce" Nov 03 22:00:00.943088 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.943068 6524 scope.go:122] "RemoveContainer" containerID="4a48814aeb26f05ca436d6300a258f42e728939c47bb85f7fbb0ae1fafa98bce" Nov 03 22:00:00.943520 ip-10-0-137-7 kubenswrapper[6524]: E1103 22:00:00.943423 6524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a48814aeb26f05ca436d6300a258f42e728939c47bb85f7fbb0ae1fafa98bce\": container with ID starting with 4a48814aeb26f05ca436d6300a258f42e728939c47bb85f7fbb0ae1fafa98bce not found: ID does not exist" containerID="4a48814aeb26f05ca436d6300a258f42e728939c47bb85f7fbb0ae1fafa98bce" Nov 03 22:00:00.943520 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.943461 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a48814aeb26f05ca436d6300a258f42e728939c47bb85f7fbb0ae1fafa98bce"} err="failed to get container status \"4a48814aeb26f05ca436d6300a258f42e728939c47bb85f7fbb0ae1fafa98bce\": rpc error: code = NotFound desc = could not find container \"4a48814aeb26f05ca436d6300a258f42e728939c47bb85f7fbb0ae1fafa98bce\": container with ID starting with 4a48814aeb26f05ca436d6300a258f42e728939c47bb85f7fbb0ae1fafa98bce not found: ID does not exist" Nov 03 22:00:00.945039 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.945020 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv"] Nov 03 22:00:00.957211 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.957182 6524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-fc16de/simulator-w4-fc16de-kserve-7dfcd7cf8b-lfcqv"] Nov 03 22:00:00.972144 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.972077 6524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-fc2d2e23-p8dlt"] Nov 03 22:00:00.972471 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.972457 6524 cpu_manager.go:414] "containerMap: removing container" podUID="82c4e70e-4765-48d3-80d3-8e0311abfda7" containerName="istio-proxy" Nov 03 22:00:00.972523 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.972473 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="82c4e70e-4765-48d3-80d3-8e0311abfda7" containerName="istio-proxy" Nov 03 22:00:00.972523 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.972485 6524 cpu_manager.go:414] "containerMap: removing container" podUID="851596d1-02e4-42b9-bdab-19a9321a2a04" containerName="main" Nov 03 22:00:00.972523 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.972490 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="851596d1-02e4-42b9-bdab-19a9321a2a04" containerName="main" Nov 03 22:00:00.972523 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.972504 6524 cpu_manager.go:414] "containerMap: removing container" podUID="779feacc-19e1-4b9e-9968-896c178c78f8" containerName="istio-proxy" Nov 03 22:00:00.972523 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.972509 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="779feacc-19e1-4b9e-9968-896c178c78f8" containerName="istio-proxy" Nov 03 22:00:00.972661 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.972563 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="851596d1-02e4-42b9-bdab-19a9321a2a04" containerName="main" Nov 03 22:00:00.972661 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.972573 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="779feacc-19e1-4b9e-9968-896c178c78f8" containerName="istio-proxy" Nov 03 22:00:00.976916 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.976895 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fc2d2e23-p8dlt" Nov 03 22:00:00.996463 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:00.988136 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Nov 03 22:00:01.008001 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.007888 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-fc2d2e23-p8dlt"] Nov 03 22:00:01.042762 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.034787 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/779feacc-19e1-4b9e-9968-896c178c78f8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/779feacc-19e1-4b9e-9968-896c178c78f8-workload-socket\") pod \"779feacc-19e1-4b9e-9968-896c178c78f8\" (UID: \"779feacc-19e1-4b9e-9968-896c178c78f8\") " Nov 03 22:00:01.042762 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.034874 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/779feacc-19e1-4b9e-9968-896c178c78f8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/779feacc-19e1-4b9e-9968-896c178c78f8-istiod-ca-cert\") pod \"779feacc-19e1-4b9e-9968-896c178c78f8\" (UID: \"779feacc-19e1-4b9e-9968-896c178c78f8\") " Nov 03 22:00:01.042762 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.034916 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/779feacc-19e1-4b9e-9968-896c178c78f8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/779feacc-19e1-4b9e-9968-896c178c78f8-istio-podinfo\") pod \"779feacc-19e1-4b9e-9968-896c178c78f8\" (UID: \"779feacc-19e1-4b9e-9968-896c178c78f8\") " Nov 03 22:00:01.042762 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.034942 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/779feacc-19e1-4b9e-9968-896c178c78f8-istio-token\" (UniqueName: \"kubernetes.io/projected/779feacc-19e1-4b9e-9968-896c178c78f8-istio-token\") pod \"779feacc-19e1-4b9e-9968-896c178c78f8\" (UID: \"779feacc-19e1-4b9e-9968-896c178c78f8\") " Nov 03 22:00:01.042762 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.034977 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/779feacc-19e1-4b9e-9968-896c178c78f8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/779feacc-19e1-4b9e-9968-896c178c78f8-istio-data\") pod \"779feacc-19e1-4b9e-9968-896c178c78f8\" (UID: \"779feacc-19e1-4b9e-9968-896c178c78f8\") " Nov 03 22:00:01.042762 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.035016 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/779feacc-19e1-4b9e-9968-896c178c78f8-kube-api-access-f5b9w\" (UniqueName: \"kubernetes.io/projected/779feacc-19e1-4b9e-9968-896c178c78f8-kube-api-access-f5b9w\") pod \"779feacc-19e1-4b9e-9968-896c178c78f8\" (UID: \"779feacc-19e1-4b9e-9968-896c178c78f8\") " Nov 03 22:00:01.042762 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.035073 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/779feacc-19e1-4b9e-9968-896c178c78f8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/779feacc-19e1-4b9e-9968-896c178c78f8-istio-envoy\") pod \"779feacc-19e1-4b9e-9968-896c178c78f8\" (UID: \"779feacc-19e1-4b9e-9968-896c178c78f8\") " Nov 03 22:00:01.042762 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.035167 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/779feacc-19e1-4b9e-9968-896c178c78f8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/779feacc-19e1-4b9e-9968-896c178c78f8-workload-certs\") pod \"779feacc-19e1-4b9e-9968-896c178c78f8\" (UID: \"779feacc-19e1-4b9e-9968-896c178c78f8\") " Nov 03 22:00:01.042762 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.035220 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/779feacc-19e1-4b9e-9968-896c178c78f8-workload-socket" pod "779feacc-19e1-4b9e-9968-896c178c78f8" (UID: "779feacc-19e1-4b9e-9968-896c178c78f8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 22:00:01.042762 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.035233 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/779feacc-19e1-4b9e-9968-896c178c78f8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/779feacc-19e1-4b9e-9968-896c178c78f8-credential-socket\") pod \"779feacc-19e1-4b9e-9968-896c178c78f8\" (UID: \"779feacc-19e1-4b9e-9968-896c178c78f8\") " Nov 03 22:00:01.042762 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.035628 6524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/779feacc-19e1-4b9e-9968-896c178c78f8-workload-socket\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:01.042762 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.035821 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/779feacc-19e1-4b9e-9968-896c178c78f8-istio-data" pod "779feacc-19e1-4b9e-9968-896c178c78f8" (UID: "779feacc-19e1-4b9e-9968-896c178c78f8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 22:00:01.042762 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.036138 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/779feacc-19e1-4b9e-9968-896c178c78f8-istiod-ca-cert" pod "779feacc-19e1-4b9e-9968-896c178c78f8" (UID: "779feacc-19e1-4b9e-9968-896c178c78f8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 03 22:00:01.042762 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.041820 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/779feacc-19e1-4b9e-9968-896c178c78f8-credential-socket" pod "779feacc-19e1-4b9e-9968-896c178c78f8" (UID: "779feacc-19e1-4b9e-9968-896c178c78f8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 22:00:01.042762 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.042421 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/779feacc-19e1-4b9e-9968-896c178c78f8-istio-podinfo" pod "779feacc-19e1-4b9e-9968-896c178c78f8" (UID: "779feacc-19e1-4b9e-9968-896c178c78f8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 03 22:00:01.042762 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.042708 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/779feacc-19e1-4b9e-9968-896c178c78f8-workload-certs" pod "779feacc-19e1-4b9e-9968-896c178c78f8" (UID: "779feacc-19e1-4b9e-9968-896c178c78f8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 22:00:01.050950 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.050907 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/779feacc-19e1-4b9e-9968-896c178c78f8-kube-api-access-f5b9w" pod "779feacc-19e1-4b9e-9968-896c178c78f8" (UID: "779feacc-19e1-4b9e-9968-896c178c78f8"). InnerVolumeSpecName "kube-api-access-f5b9w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 03 22:00:01.051297 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.051269 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/779feacc-19e1-4b9e-9968-896c178c78f8-istio-token" pod "779feacc-19e1-4b9e-9968-896c178c78f8" (UID: "779feacc-19e1-4b9e-9968-896c178c78f8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 03 22:00:01.052413 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.052385 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/779feacc-19e1-4b9e-9968-896c178c78f8-istio-envoy" pod "779feacc-19e1-4b9e-9968-896c178c78f8" (UID: "779feacc-19e1-4b9e-9968-896c178c78f8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 22:00:01.136964 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.136923 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dtvt\" (UniqueName: \"kubernetes.io/projected/b25782b8-1123-4fb1-9d07-1f121843c479-kube-api-access-6dtvt\") pod \"maas-api-revoke-sub-fc2d2e23-p8dlt\" (UID: \"b25782b8-1123-4fb1-9d07-1f121843c479\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fc2d2e23-p8dlt" Nov 03 22:00:01.137362 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.136973 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b25782b8-1123-4fb1-9d07-1f121843c479-maas-api-xks-ca\") pod \"maas-api-revoke-sub-fc2d2e23-p8dlt\" (UID: \"b25782b8-1123-4fb1-9d07-1f121843c479\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fc2d2e23-p8dlt" Nov 03 22:00:01.137362 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.136994 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b25782b8-1123-4fb1-9d07-1f121843c479-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-fc2d2e23-p8dlt\" (UID: \"b25782b8-1123-4fb1-9d07-1f121843c479\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fc2d2e23-p8dlt" Nov 03 22:00:01.137362 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.137144 6524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/779feacc-19e1-4b9e-9968-896c178c78f8-istiod-ca-cert\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:01.137362 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.137166 6524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/779feacc-19e1-4b9e-9968-896c178c78f8-istio-podinfo\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:01.137362 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.137175 6524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/779feacc-19e1-4b9e-9968-896c178c78f8-istio-token\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:01.137362 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.137185 6524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/779feacc-19e1-4b9e-9968-896c178c78f8-istio-data\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:01.137362 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.137194 6524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f5b9w\" (UniqueName: \"kubernetes.io/projected/779feacc-19e1-4b9e-9968-896c178c78f8-kube-api-access-f5b9w\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:01.137362 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.137203 6524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/779feacc-19e1-4b9e-9968-896c178c78f8-istio-envoy\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:01.137362 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.137211 6524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/779feacc-19e1-4b9e-9968-896c178c78f8-workload-certs\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:01.137362 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.137220 6524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/779feacc-19e1-4b9e-9968-896c178c78f8-credential-socket\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:01.238086 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.237996 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dtvt\" (UniqueName: \"kubernetes.io/projected/b25782b8-1123-4fb1-9d07-1f121843c479-kube-api-access-6dtvt\") pod \"maas-api-revoke-sub-fc2d2e23-p8dlt\" (UID: \"b25782b8-1123-4fb1-9d07-1f121843c479\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fc2d2e23-p8dlt" Nov 03 22:00:01.238086 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.238041 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b25782b8-1123-4fb1-9d07-1f121843c479-maas-api-xks-ca\") pod \"maas-api-revoke-sub-fc2d2e23-p8dlt\" (UID: \"b25782b8-1123-4fb1-9d07-1f121843c479\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fc2d2e23-p8dlt" Nov 03 22:00:01.238086 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.238060 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b25782b8-1123-4fb1-9d07-1f121843c479-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-fc2d2e23-p8dlt\" (UID: \"b25782b8-1123-4fb1-9d07-1f121843c479\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fc2d2e23-p8dlt" Nov 03 22:00:01.238822 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.238795 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b25782b8-1123-4fb1-9d07-1f121843c479-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-fc2d2e23-p8dlt\" (UID: \"b25782b8-1123-4fb1-9d07-1f121843c479\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fc2d2e23-p8dlt" Nov 03 22:00:01.240348 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.240326 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b25782b8-1123-4fb1-9d07-1f121843c479-maas-api-xks-ca\") pod \"maas-api-revoke-sub-fc2d2e23-p8dlt\" (UID: \"b25782b8-1123-4fb1-9d07-1f121843c479\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fc2d2e23-p8dlt" Nov 03 22:00:01.250958 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.250927 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dtvt\" (UniqueName: \"kubernetes.io/projected/b25782b8-1123-4fb1-9d07-1f121843c479-kube-api-access-6dtvt\") pod \"maas-api-revoke-sub-fc2d2e23-p8dlt\" (UID: \"b25782b8-1123-4fb1-9d07-1f121843c479\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fc2d2e23-p8dlt" Nov 03 22:00:01.280455 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.280410 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9"] Nov 03 22:00:01.286905 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.286872 6524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-8af3f3-openshift-default-56b456db75-rw6g9"] Nov 03 22:00:01.335381 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.335345 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fc2d2e23-p8dlt" Nov 03 22:00:01.653467 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.653439 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-fc2d2e23-p8dlt"] Nov 03 22:00:01.655015 ip-10-0-137-7 kubenswrapper[6524]: W1103 22:00:01.654987 6524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb25782b8_1123_4fb1_9d07_1f121843c479.slice/crio-1e43d406d8e890528f05763b9a7a401b86df2512d12358ea0a6e9f9b40e88718 WatchSource:0}: Error finding container 1e43d406d8e890528f05763b9a7a401b86df2512d12358ea0a6e9f9b40e88718: Status 404 returned error can't find the container with id 1e43d406d8e890528f05763b9a7a401b86df2512d12358ea0a6e9f9b40e88718 Nov 03 22:00:01.940423 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.940200 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fc2d2e23-p8dlt" event={"ID":"b25782b8-1123-4fb1-9d07-1f121843c479","Type":"ContainerStarted","Data":"1e43d406d8e890528f05763b9a7a401b86df2512d12358ea0a6e9f9b40e88718"} Nov 03 22:00:01.942366 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.942334 6524 generic.go:356] "Generic (PLEG): container finished" podID="8462643c-89e1-4ee3-a413-44160da8e03d" containerID="3e1d409fe806d4bbaba0b8b39ab4c5c332583086ee3c8a99cb87a9fc7796744e" exitCode=0 Nov 03 22:00:01.942496 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.942416 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" event={"ID":"8462643c-89e1-4ee3-a413-44160da8e03d","Type":"ContainerDied","Data":"3e1d409fe806d4bbaba0b8b39ab4c5c332583086ee3c8a99cb87a9fc7796744e"} Nov 03 22:00:01.944787 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.944754 6524 generic.go:356] "Generic (PLEG): container finished" podID="587e7995-5cb6-4f61-9f3e-a3fe113a39ad" containerID="5a0a85adb5005df91bec0873778c353668fd091daefacc15b25972df8eb774c4" exitCode=60 Nov 03 22:00:01.944903 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.944820 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-fc16de-29895720-kgtn8" event={"ID":"587e7995-5cb6-4f61-9f3e-a3fe113a39ad","Type":"ContainerDied","Data":"5a0a85adb5005df91bec0873778c353668fd091daefacc15b25972df8eb774c4"} Nov 03 22:00:01.945712 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:01.945234 6524 scope.go:122] "RemoveContainer" containerID="5a0a85adb5005df91bec0873778c353668fd091daefacc15b25972df8eb774c4" Nov 03 22:00:02.058805 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:02.058770 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" Nov 03 22:00:02.147791 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:02.147756 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8462643c-89e1-4ee3-a413-44160da8e03d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8462643c-89e1-4ee3-a413-44160da8e03d-istio-podinfo\") pod \"8462643c-89e1-4ee3-a413-44160da8e03d\" (UID: \"8462643c-89e1-4ee3-a413-44160da8e03d\") " Nov 03 22:00:02.147791 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:02.147807 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8462643c-89e1-4ee3-a413-44160da8e03d-istio-token\" (UniqueName: \"kubernetes.io/projected/8462643c-89e1-4ee3-a413-44160da8e03d-istio-token\") pod \"8462643c-89e1-4ee3-a413-44160da8e03d\" (UID: \"8462643c-89e1-4ee3-a413-44160da8e03d\") " Nov 03 22:00:02.148352 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:02.147843 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8462643c-89e1-4ee3-a413-44160da8e03d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8462643c-89e1-4ee3-a413-44160da8e03d-istio-data\") pod \"8462643c-89e1-4ee3-a413-44160da8e03d\" (UID: \"8462643c-89e1-4ee3-a413-44160da8e03d\") " Nov 03 22:00:02.148352 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:02.147900 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8462643c-89e1-4ee3-a413-44160da8e03d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8462643c-89e1-4ee3-a413-44160da8e03d-workload-socket\") pod \"8462643c-89e1-4ee3-a413-44160da8e03d\" (UID: \"8462643c-89e1-4ee3-a413-44160da8e03d\") " Nov 03 22:00:02.148352 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:02.147932 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8462643c-89e1-4ee3-a413-44160da8e03d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8462643c-89e1-4ee3-a413-44160da8e03d-istiod-ca-cert\") pod \"8462643c-89e1-4ee3-a413-44160da8e03d\" (UID: \"8462643c-89e1-4ee3-a413-44160da8e03d\") " Nov 03 22:00:02.148352 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:02.147977 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8462643c-89e1-4ee3-a413-44160da8e03d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8462643c-89e1-4ee3-a413-44160da8e03d-workload-certs\") pod \"8462643c-89e1-4ee3-a413-44160da8e03d\" (UID: \"8462643c-89e1-4ee3-a413-44160da8e03d\") " Nov 03 22:00:02.148352 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:02.148016 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8462643c-89e1-4ee3-a413-44160da8e03d-kube-api-access-26lhf\" (UniqueName: \"kubernetes.io/projected/8462643c-89e1-4ee3-a413-44160da8e03d-kube-api-access-26lhf\") pod \"8462643c-89e1-4ee3-a413-44160da8e03d\" (UID: \"8462643c-89e1-4ee3-a413-44160da8e03d\") " Nov 03 22:00:02.148352 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:02.148090 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8462643c-89e1-4ee3-a413-44160da8e03d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8462643c-89e1-4ee3-a413-44160da8e03d-credential-socket\") pod \"8462643c-89e1-4ee3-a413-44160da8e03d\" (UID: \"8462643c-89e1-4ee3-a413-44160da8e03d\") " Nov 03 22:00:02.148352 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:02.148213 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8462643c-89e1-4ee3-a413-44160da8e03d-istio-data" pod "8462643c-89e1-4ee3-a413-44160da8e03d" (UID: "8462643c-89e1-4ee3-a413-44160da8e03d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 22:00:02.148352 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:02.148226 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8462643c-89e1-4ee3-a413-44160da8e03d-workload-socket" pod "8462643c-89e1-4ee3-a413-44160da8e03d" (UID: "8462643c-89e1-4ee3-a413-44160da8e03d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 22:00:02.148352 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:02.148239 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8462643c-89e1-4ee3-a413-44160da8e03d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8462643c-89e1-4ee3-a413-44160da8e03d-istio-envoy\") pod \"8462643c-89e1-4ee3-a413-44160da8e03d\" (UID: \"8462643c-89e1-4ee3-a413-44160da8e03d\") " Nov 03 22:00:02.148352 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:02.148333 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8462643c-89e1-4ee3-a413-44160da8e03d-workload-certs" pod "8462643c-89e1-4ee3-a413-44160da8e03d" (UID: "8462643c-89e1-4ee3-a413-44160da8e03d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 22:00:02.148820 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:02.148447 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8462643c-89e1-4ee3-a413-44160da8e03d-credential-socket" pod "8462643c-89e1-4ee3-a413-44160da8e03d" (UID: "8462643c-89e1-4ee3-a413-44160da8e03d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 22:00:02.148820 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:02.148489 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8462643c-89e1-4ee3-a413-44160da8e03d-istiod-ca-cert" pod "8462643c-89e1-4ee3-a413-44160da8e03d" (UID: "8462643c-89e1-4ee3-a413-44160da8e03d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 03 22:00:02.148820 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:02.148674 6524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8462643c-89e1-4ee3-a413-44160da8e03d-istio-data\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:02.148820 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:02.148694 6524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8462643c-89e1-4ee3-a413-44160da8e03d-workload-socket\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:02.148820 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:02.148711 6524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8462643c-89e1-4ee3-a413-44160da8e03d-istiod-ca-cert\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:02.148820 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:02.148726 6524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8462643c-89e1-4ee3-a413-44160da8e03d-workload-certs\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:02.148820 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:02.148740 6524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8462643c-89e1-4ee3-a413-44160da8e03d-credential-socket\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:02.186049 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:02.185964 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8462643c-89e1-4ee3-a413-44160da8e03d-kube-api-access-26lhf" pod "8462643c-89e1-4ee3-a413-44160da8e03d" (UID: "8462643c-89e1-4ee3-a413-44160da8e03d"). InnerVolumeSpecName "kube-api-access-26lhf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 03 22:00:02.186049 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:02.186008 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8462643c-89e1-4ee3-a413-44160da8e03d-istio-envoy" pod "8462643c-89e1-4ee3-a413-44160da8e03d" (UID: "8462643c-89e1-4ee3-a413-44160da8e03d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 22:00:02.186049 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:02.185964 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8462643c-89e1-4ee3-a413-44160da8e03d-istio-podinfo" pod "8462643c-89e1-4ee3-a413-44160da8e03d" (UID: "8462643c-89e1-4ee3-a413-44160da8e03d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 03 22:00:02.186656 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:02.186588 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8462643c-89e1-4ee3-a413-44160da8e03d-istio-token" pod "8462643c-89e1-4ee3-a413-44160da8e03d" (UID: "8462643c-89e1-4ee3-a413-44160da8e03d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 03 22:00:02.249811 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:02.249772 6524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26lhf\" (UniqueName: \"kubernetes.io/projected/8462643c-89e1-4ee3-a413-44160da8e03d-kube-api-access-26lhf\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:02.249811 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:02.249814 6524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8462643c-89e1-4ee3-a413-44160da8e03d-istio-envoy\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:02.250024 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:02.249829 6524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8462643c-89e1-4ee3-a413-44160da8e03d-istio-podinfo\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:02.250024 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:02.249843 6524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8462643c-89e1-4ee3-a413-44160da8e03d-istio-token\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:02.767844 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:02.767807 6524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="779feacc-19e1-4b9e-9968-896c178c78f8" path="/var/lib/kubelet/pods/779feacc-19e1-4b9e-9968-896c178c78f8/volumes" Nov 03 22:00:02.768216 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:02.768202 6524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="851596d1-02e4-42b9-bdab-19a9321a2a04" path="/var/lib/kubelet/pods/851596d1-02e4-42b9-bdab-19a9321a2a04/volumes" Nov 03 22:00:02.951106 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:02.951069 6524 generic.go:356] "Generic (PLEG): container finished" podID="b25782b8-1123-4fb1-9d07-1f121843c479" containerID="22f74bb07d5e4ce1c7b8cbde113ee4af59c8b04eadfa27d9390b45e6ffb727e1" exitCode=0 Nov 03 22:00:02.951324 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:02.951179 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fc2d2e23-p8dlt" event={"ID":"b25782b8-1123-4fb1-9d07-1f121843c479","Type":"ContainerDied","Data":"22f74bb07d5e4ce1c7b8cbde113ee4af59c8b04eadfa27d9390b45e6ffb727e1"} Nov 03 22:00:02.952669 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:02.952646 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" Nov 03 22:00:02.952809 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:02.952645 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9" event={"ID":"8462643c-89e1-4ee3-a413-44160da8e03d","Type":"ContainerDied","Data":"bdb7039e3dfd64fdfaa52141d5e4680d1b28b96bf9d0988322e42ee644881e5f"} Nov 03 22:00:02.952809 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:02.952779 6524 scope.go:122] "RemoveContainer" containerID="3e1d409fe806d4bbaba0b8b39ab4c5c332583086ee3c8a99cb87a9fc7796744e" Nov 03 22:00:02.954787 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:02.954759 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-fc16de-29895720-kgtn8" event={"ID":"587e7995-5cb6-4f61-9f3e-a3fe113a39ad","Type":"ContainerStarted","Data":"1f16d5730f72808c85e0f1cfe4da349bc4552562e6d709d8dbeff2123a1bed28"} Nov 03 22:00:02.991461 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:02.991429 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9"] Nov 03 22:00:02.995331 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:02.995304 6524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-02e8ab-openshift-default-6f4974886c-pnhp9"] Nov 03 22:00:03.010951 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:03.010895 6524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-fc16de-29895720-kgtn8" podStartSLOduration=3.010880851 podStartE2EDuration="3.010880851s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:00:03.009940751 +0000 UTC m=+848.362267556" watchObservedRunningTime="2026-11-03 22:00:03.010880851 +0000 UTC m=+848.363207654" Nov 03 22:00:03.963998 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:03.963961 6524 generic.go:356] "Generic (PLEG): container finished" podID="587e7995-5cb6-4f61-9f3e-a3fe113a39ad" containerID="1f16d5730f72808c85e0f1cfe4da349bc4552562e6d709d8dbeff2123a1bed28" exitCode=60 Nov 03 22:00:03.964450 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:03.964033 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-fc16de-29895720-kgtn8" event={"ID":"587e7995-5cb6-4f61-9f3e-a3fe113a39ad","Type":"ContainerDied","Data":"1f16d5730f72808c85e0f1cfe4da349bc4552562e6d709d8dbeff2123a1bed28"} Nov 03 22:00:03.964450 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:03.964080 6524 scope.go:122] "RemoveContainer" containerID="5a0a85adb5005df91bec0873778c353668fd091daefacc15b25972df8eb774c4" Nov 03 22:00:03.964450 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:03.964328 6524 scope.go:122] "RemoveContainer" containerID="1f16d5730f72808c85e0f1cfe4da349bc4552562e6d709d8dbeff2123a1bed28" Nov 03 22:00:03.964891 ip-10-0-137-7 kubenswrapper[6524]: E1103 22:00:03.964572 6524 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-fc16de-29895720-kgtn8_odh-ai-gateway-infra(587e7995-5cb6-4f61-9f3e-a3fe113a39ad)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-fc16de-29895720-kgtn8" podUID="587e7995-5cb6-4f61-9f3e-a3fe113a39ad" Nov 03 22:00:04.140518 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:04.140490 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fc2d2e23-p8dlt" Nov 03 22:00:04.269410 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:04.269321 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b25782b8-1123-4fb1-9d07-1f121843c479-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b25782b8-1123-4fb1-9d07-1f121843c479-maas-api-openshift-ca\") pod \"b25782b8-1123-4fb1-9d07-1f121843c479\" (UID: \"b25782b8-1123-4fb1-9d07-1f121843c479\") " Nov 03 22:00:04.269607 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:04.269433 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b25782b8-1123-4fb1-9d07-1f121843c479-kube-api-access-6dtvt\" (UniqueName: \"kubernetes.io/projected/b25782b8-1123-4fb1-9d07-1f121843c479-kube-api-access-6dtvt\") pod \"b25782b8-1123-4fb1-9d07-1f121843c479\" (UID: \"b25782b8-1123-4fb1-9d07-1f121843c479\") " Nov 03 22:00:04.269607 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:04.269503 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b25782b8-1123-4fb1-9d07-1f121843c479-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b25782b8-1123-4fb1-9d07-1f121843c479-maas-api-xks-ca\") pod \"b25782b8-1123-4fb1-9d07-1f121843c479\" (UID: \"b25782b8-1123-4fb1-9d07-1f121843c479\") " Nov 03 22:00:04.269753 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:04.269665 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b25782b8-1123-4fb1-9d07-1f121843c479-maas-api-openshift-ca" pod "b25782b8-1123-4fb1-9d07-1f121843c479" (UID: "b25782b8-1123-4fb1-9d07-1f121843c479"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 03 22:00:04.269872 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:04.269853 6524 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b25782b8-1123-4fb1-9d07-1f121843c479-maas-api-openshift-ca\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:04.271545 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:04.271519 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b25782b8-1123-4fb1-9d07-1f121843c479-maas-api-xks-ca" pod "b25782b8-1123-4fb1-9d07-1f121843c479" (UID: "b25782b8-1123-4fb1-9d07-1f121843c479"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 03 22:00:04.271661 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:04.271644 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b25782b8-1123-4fb1-9d07-1f121843c479-kube-api-access-6dtvt" pod "b25782b8-1123-4fb1-9d07-1f121843c479" (UID: "b25782b8-1123-4fb1-9d07-1f121843c479"). InnerVolumeSpecName "kube-api-access-6dtvt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 03 22:00:04.370676 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:04.370643 6524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dtvt\" (UniqueName: \"kubernetes.io/projected/b25782b8-1123-4fb1-9d07-1f121843c479-kube-api-access-6dtvt\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:04.370676 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:04.370674 6524 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b25782b8-1123-4fb1-9d07-1f121843c479-maas-api-xks-ca\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:04.609561 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:04.609514 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-fc16de-29895720-kgtn8"] Nov 03 22:00:04.768687 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:04.768644 6524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8462643c-89e1-4ee3-a413-44160da8e03d" path="/var/lib/kubelet/pods/8462643c-89e1-4ee3-a413-44160da8e03d/volumes" Nov 03 22:00:04.971937 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:04.971845 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fc2d2e23-p8dlt" Nov 03 22:00:04.972456 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:04.972427 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fc2d2e23-p8dlt" event={"ID":"b25782b8-1123-4fb1-9d07-1f121843c479","Type":"ContainerDied","Data":"1e43d406d8e890528f05763b9a7a401b86df2512d12358ea0a6e9f9b40e88718"} Nov 03 22:00:04.972542 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:04.972487 6524 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1e43d406d8e890528f05763b9a7a401b86df2512d12358ea0a6e9f9b40e88718" Nov 03 22:00:04.973984 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:04.973960 6524 scope.go:122] "RemoveContainer" containerID="1f16d5730f72808c85e0f1cfe4da349bc4552562e6d709d8dbeff2123a1bed28" Nov 03 22:00:04.974282 ip-10-0-137-7 kubenswrapper[6524]: E1103 22:00:04.974260 6524 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-fc16de-29895720-kgtn8_odh-ai-gateway-infra(587e7995-5cb6-4f61-9f3e-a3fe113a39ad)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-fc16de-29895720-kgtn8" podUID="587e7995-5cb6-4f61-9f3e-a3fe113a39ad" Nov 03 22:00:06.121849 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:06.121818 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-fc16de-29895720-kgtn8" Nov 03 22:00:06.298255 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:06.298158 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/587e7995-5cb6-4f61-9f3e-a3fe113a39ad-kube-api-access-hhjnm\" (UniqueName: \"kubernetes.io/projected/587e7995-5cb6-4f61-9f3e-a3fe113a39ad-kube-api-access-hhjnm\") pod \"587e7995-5cb6-4f61-9f3e-a3fe113a39ad\" (UID: \"587e7995-5cb6-4f61-9f3e-a3fe113a39ad\") " Nov 03 22:00:06.298418 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:06.298304 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/587e7995-5cb6-4f61-9f3e-a3fe113a39ad-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/587e7995-5cb6-4f61-9f3e-a3fe113a39ad-maas-api-service-ca\") pod \"587e7995-5cb6-4f61-9f3e-a3fe113a39ad\" (UID: \"587e7995-5cb6-4f61-9f3e-a3fe113a39ad\") " Nov 03 22:00:06.298660 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:06.298628 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/587e7995-5cb6-4f61-9f3e-a3fe113a39ad-maas-api-service-ca" pod "587e7995-5cb6-4f61-9f3e-a3fe113a39ad" (UID: "587e7995-5cb6-4f61-9f3e-a3fe113a39ad"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 03 22:00:06.300324 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:06.300293 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/587e7995-5cb6-4f61-9f3e-a3fe113a39ad-kube-api-access-hhjnm" pod "587e7995-5cb6-4f61-9f3e-a3fe113a39ad" (UID: "587e7995-5cb6-4f61-9f3e-a3fe113a39ad"). InnerVolumeSpecName "kube-api-access-hhjnm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 03 22:00:06.399392 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:06.399355 6524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhjnm\" (UniqueName: \"kubernetes.io/projected/587e7995-5cb6-4f61-9f3e-a3fe113a39ad-kube-api-access-hhjnm\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:06.399392 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:06.399387 6524 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/587e7995-5cb6-4f61-9f3e-a3fe113a39ad-maas-api-service-ca\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:06.986204 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:06.986160 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-fc16de-29895720-kgtn8" event={"ID":"587e7995-5cb6-4f61-9f3e-a3fe113a39ad","Type":"ContainerDied","Data":"6df65979af349819dc12ff2563675a7348ee6645348a44afa56fbe8fe1ec6ed1"} Nov 03 22:00:06.986432 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:06.986184 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-fc16de-29895720-kgtn8" Nov 03 22:00:06.986432 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:06.986274 6524 scope.go:122] "RemoveContainer" containerID="1f16d5730f72808c85e0f1cfe4da349bc4552562e6d709d8dbeff2123a1bed28" Nov 03 22:00:07.005369 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:07.005327 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-fc16de-29895720-kgtn8"] Nov 03 22:00:07.011860 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:07.011826 6524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-fc16de-29895720-kgtn8"] Nov 03 22:00:08.766848 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:08.766812 6524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="587e7995-5cb6-4f61-9f3e-a3fe113a39ad" path="/var/lib/kubelet/pods/587e7995-5cb6-4f61-9f3e-a3fe113a39ad/volumes" Nov 03 22:00:12.024931 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:12.024887 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45"] Nov 03 22:00:12.025380 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:12.025166 6524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" podUID="a9593fbc-9af2-4c1f-a986-3dc48dab11a7" containerName="istio-proxy" containerID="cri-o://99e28e6f8cf025b81892cd5b47a26086c85b17189fb870aa1076caf8f3b98988" gracePeriod=30 Nov 03 22:00:17.292857 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:17.292797 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" Nov 03 22:00:17.338691 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:17.338645 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-istio-envoy\") pod \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\" (UID: \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\") " Nov 03 22:00:17.338890 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:17.338711 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-workload-socket\") pod \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\" (UID: \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\") " Nov 03 22:00:17.338890 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:17.338780 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-istio-data\") pod \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\" (UID: \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\") " Nov 03 22:00:17.338890 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:17.338817 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-istio-token\" (UniqueName: \"kubernetes.io/projected/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-istio-token\") pod \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\" (UID: \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\") " Nov 03 22:00:17.338890 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:17.338846 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-istiod-ca-cert\") pod \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\" (UID: \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\") " Nov 03 22:00:17.339116 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:17.338903 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-kube-api-access-qvv9x\" (UniqueName: \"kubernetes.io/projected/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-kube-api-access-qvv9x\") pod \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\" (UID: \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\") " Nov 03 22:00:17.339116 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:17.338957 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-credential-socket\") pod \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\" (UID: \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\") " Nov 03 22:00:17.339116 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:17.338990 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-workload-certs\") pod \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\" (UID: \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\") " Nov 03 22:00:17.339116 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:17.339079 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-istio-podinfo\") pod \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\" (UID: \"a9593fbc-9af2-4c1f-a986-3dc48dab11a7\") " Nov 03 22:00:17.340110 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:17.339547 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-istio-data" pod "a9593fbc-9af2-4c1f-a986-3dc48dab11a7" (UID: "a9593fbc-9af2-4c1f-a986-3dc48dab11a7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 22:00:17.340110 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:17.339692 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-workload-certs" pod "a9593fbc-9af2-4c1f-a986-3dc48dab11a7" (UID: "a9593fbc-9af2-4c1f-a986-3dc48dab11a7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 22:00:17.340110 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:17.339771 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-credential-socket" pod "a9593fbc-9af2-4c1f-a986-3dc48dab11a7" (UID: "a9593fbc-9af2-4c1f-a986-3dc48dab11a7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 22:00:17.340110 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:17.339890 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-istiod-ca-cert" pod "a9593fbc-9af2-4c1f-a986-3dc48dab11a7" (UID: "a9593fbc-9af2-4c1f-a986-3dc48dab11a7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 03 22:00:17.340110 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:17.339927 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-workload-socket" pod "a9593fbc-9af2-4c1f-a986-3dc48dab11a7" (UID: "a9593fbc-9af2-4c1f-a986-3dc48dab11a7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 22:00:17.342728 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:17.342597 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-istio-envoy" pod "a9593fbc-9af2-4c1f-a986-3dc48dab11a7" (UID: "a9593fbc-9af2-4c1f-a986-3dc48dab11a7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 22:00:17.342728 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:17.342685 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-istio-token" pod "a9593fbc-9af2-4c1f-a986-3dc48dab11a7" (UID: "a9593fbc-9af2-4c1f-a986-3dc48dab11a7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 03 22:00:17.342890 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:17.342800 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-kube-api-access-qvv9x" pod "a9593fbc-9af2-4c1f-a986-3dc48dab11a7" (UID: "a9593fbc-9af2-4c1f-a986-3dc48dab11a7"). InnerVolumeSpecName "kube-api-access-qvv9x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 03 22:00:17.343003 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:17.342980 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-istio-podinfo" pod "a9593fbc-9af2-4c1f-a986-3dc48dab11a7" (UID: "a9593fbc-9af2-4c1f-a986-3dc48dab11a7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 03 22:00:17.440227 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:17.440184 6524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-workload-socket\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:17.440227 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:17.440217 6524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-istio-data\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:17.440227 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:17.440227 6524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-istio-token\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:17.440227 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:17.440239 6524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-istiod-ca-cert\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:17.440600 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:17.440250 6524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvv9x\" (UniqueName: \"kubernetes.io/projected/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-kube-api-access-qvv9x\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:17.440600 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:17.440259 6524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-credential-socket\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:17.440600 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:17.440270 6524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-workload-certs\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:17.440600 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:17.440278 6524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-istio-podinfo\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:17.440600 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:17.440287 6524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9593fbc-9af2-4c1f-a986-3dc48dab11a7-istio-envoy\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:18.048568 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:18.048529 6524 generic.go:356] "Generic (PLEG): container finished" podID="a9593fbc-9af2-4c1f-a986-3dc48dab11a7" containerID="99e28e6f8cf025b81892cd5b47a26086c85b17189fb870aa1076caf8f3b98988" exitCode=0 Nov 03 22:00:18.048752 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:18.048583 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" Nov 03 22:00:18.048752 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:18.048584 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" event={"ID":"a9593fbc-9af2-4c1f-a986-3dc48dab11a7","Type":"ContainerDied","Data":"99e28e6f8cf025b81892cd5b47a26086c85b17189fb870aa1076caf8f3b98988"} Nov 03 22:00:18.048752 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:18.048704 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" event={"ID":"a9593fbc-9af2-4c1f-a986-3dc48dab11a7","Type":"ContainerDied","Data":"f77c00b7ba1e9c2ae7e80a2267b0d83c3bd9df74329d22eb673d8681b0320efa"} Nov 03 22:00:18.048752 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:18.048732 6524 scope.go:122] "RemoveContainer" containerID="99e28e6f8cf025b81892cd5b47a26086c85b17189fb870aa1076caf8f3b98988" Nov 03 22:00:18.058280 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:18.058257 6524 scope.go:122] "RemoveContainer" containerID="99e28e6f8cf025b81892cd5b47a26086c85b17189fb870aa1076caf8f3b98988" Nov 03 22:00:18.058590 ip-10-0-137-7 kubenswrapper[6524]: E1103 22:00:18.058570 6524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99e28e6f8cf025b81892cd5b47a26086c85b17189fb870aa1076caf8f3b98988\": container with ID starting with 99e28e6f8cf025b81892cd5b47a26086c85b17189fb870aa1076caf8f3b98988 not found: ID does not exist" containerID="99e28e6f8cf025b81892cd5b47a26086c85b17189fb870aa1076caf8f3b98988" Nov 03 22:00:18.058656 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:18.058599 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99e28e6f8cf025b81892cd5b47a26086c85b17189fb870aa1076caf8f3b98988"} err="failed to get container status \"99e28e6f8cf025b81892cd5b47a26086c85b17189fb870aa1076caf8f3b98988\": rpc error: code = NotFound desc = could not find container \"99e28e6f8cf025b81892cd5b47a26086c85b17189fb870aa1076caf8f3b98988\": container with ID starting with 99e28e6f8cf025b81892cd5b47a26086c85b17189fb870aa1076caf8f3b98988 not found: ID does not exist" Nov 03 22:00:18.071876 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:18.071850 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45"] Nov 03 22:00:18.076384 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:18.076362 6524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45"] Nov 03 22:00:18.238318 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:18.238281 6524 patch_prober.go:29] interesting pod/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 03 22:00:18.238492 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:18.238353 6524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-fc16de-openshift-default-85fdcb545f-4tm45" podUID="a9593fbc-9af2-4c1f-a986-3dc48dab11a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 03 22:00:18.766277 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:18.766238 6524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a9593fbc-9af2-4c1f-a986-3dc48dab11a7" path="/var/lib/kubelet/pods/a9593fbc-9af2-4c1f-a986-3dc48dab11a7/volumes" Nov 03 22:00:22.635805 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:22.635763 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd"] Nov 03 22:00:22.636822 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:22.636068 6524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" podUID="1d3e9770-476c-47a5-91d7-afffb476b8c7" containerName="istio-proxy" containerID="cri-o://9c2ab59138d7016737610ce46a8b1def9de74b3e058362dcacd9fab67f6178a9" gracePeriod=30 Nov 03 22:00:28.142940 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:28.142843 6524 generic.go:356] "Generic (PLEG): container finished" podID="1d3e9770-476c-47a5-91d7-afffb476b8c7" containerID="9c2ab59138d7016737610ce46a8b1def9de74b3e058362dcacd9fab67f6178a9" exitCode=0 Nov 03 22:00:28.143594 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:28.143538 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" event={"ID":"1d3e9770-476c-47a5-91d7-afffb476b8c7","Type":"ContainerDied","Data":"9c2ab59138d7016737610ce46a8b1def9de74b3e058362dcacd9fab67f6178a9"} Nov 03 22:00:28.181160 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:28.180399 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" Nov 03 22:00:28.273863 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:28.273773 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d3e9770-476c-47a5-91d7-afffb476b8c7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d3e9770-476c-47a5-91d7-afffb476b8c7-istio-data\") pod \"1d3e9770-476c-47a5-91d7-afffb476b8c7\" (UID: \"1d3e9770-476c-47a5-91d7-afffb476b8c7\") " Nov 03 22:00:28.274105 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:28.274082 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1d3e9770-476c-47a5-91d7-afffb476b8c7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d3e9770-476c-47a5-91d7-afffb476b8c7-istio-podinfo\") pod \"1d3e9770-476c-47a5-91d7-afffb476b8c7\" (UID: \"1d3e9770-476c-47a5-91d7-afffb476b8c7\") " Nov 03 22:00:28.274281 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:28.274263 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d3e9770-476c-47a5-91d7-afffb476b8c7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d3e9770-476c-47a5-91d7-afffb476b8c7-istio-envoy\") pod \"1d3e9770-476c-47a5-91d7-afffb476b8c7\" (UID: \"1d3e9770-476c-47a5-91d7-afffb476b8c7\") " Nov 03 22:00:28.274419 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:28.274403 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d3e9770-476c-47a5-91d7-afffb476b8c7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d3e9770-476c-47a5-91d7-afffb476b8c7-workload-certs\") pod \"1d3e9770-476c-47a5-91d7-afffb476b8c7\" (UID: \"1d3e9770-476c-47a5-91d7-afffb476b8c7\") " Nov 03 22:00:28.274555 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:28.274538 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1d3e9770-476c-47a5-91d7-afffb476b8c7-istio-token\" (UniqueName: \"kubernetes.io/projected/1d3e9770-476c-47a5-91d7-afffb476b8c7-istio-token\") pod \"1d3e9770-476c-47a5-91d7-afffb476b8c7\" (UID: \"1d3e9770-476c-47a5-91d7-afffb476b8c7\") " Nov 03 22:00:28.274680 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:28.274663 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d3e9770-476c-47a5-91d7-afffb476b8c7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d3e9770-476c-47a5-91d7-afffb476b8c7-credential-socket\") pod \"1d3e9770-476c-47a5-91d7-afffb476b8c7\" (UID: \"1d3e9770-476c-47a5-91d7-afffb476b8c7\") " Nov 03 22:00:28.274840 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:28.274823 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1d3e9770-476c-47a5-91d7-afffb476b8c7-kube-api-access-fwrcl\" (UniqueName: \"kubernetes.io/projected/1d3e9770-476c-47a5-91d7-afffb476b8c7-kube-api-access-fwrcl\") pod \"1d3e9770-476c-47a5-91d7-afffb476b8c7\" (UID: \"1d3e9770-476c-47a5-91d7-afffb476b8c7\") " Nov 03 22:00:28.275009 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:28.274993 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d3e9770-476c-47a5-91d7-afffb476b8c7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d3e9770-476c-47a5-91d7-afffb476b8c7-workload-socket\") pod \"1d3e9770-476c-47a5-91d7-afffb476b8c7\" (UID: \"1d3e9770-476c-47a5-91d7-afffb476b8c7\") " Nov 03 22:00:28.275157 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:28.275141 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1d3e9770-476c-47a5-91d7-afffb476b8c7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d3e9770-476c-47a5-91d7-afffb476b8c7-istiod-ca-cert\") pod \"1d3e9770-476c-47a5-91d7-afffb476b8c7\" (UID: \"1d3e9770-476c-47a5-91d7-afffb476b8c7\") " Nov 03 22:00:28.275936 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:28.275915 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d3e9770-476c-47a5-91d7-afffb476b8c7-istiod-ca-cert" pod "1d3e9770-476c-47a5-91d7-afffb476b8c7" (UID: "1d3e9770-476c-47a5-91d7-afffb476b8c7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 03 22:00:28.276363 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:28.276343 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d3e9770-476c-47a5-91d7-afffb476b8c7-istio-data" pod "1d3e9770-476c-47a5-91d7-afffb476b8c7" (UID: "1d3e9770-476c-47a5-91d7-afffb476b8c7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 22:00:28.277378 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:28.277048 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d3e9770-476c-47a5-91d7-afffb476b8c7-credential-socket" pod "1d3e9770-476c-47a5-91d7-afffb476b8c7" (UID: "1d3e9770-476c-47a5-91d7-afffb476b8c7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 22:00:28.277480 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:28.277396 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d3e9770-476c-47a5-91d7-afffb476b8c7-workload-socket" pod "1d3e9770-476c-47a5-91d7-afffb476b8c7" (UID: "1d3e9770-476c-47a5-91d7-afffb476b8c7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 22:00:28.279200 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:28.279173 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d3e9770-476c-47a5-91d7-afffb476b8c7-workload-certs" pod "1d3e9770-476c-47a5-91d7-afffb476b8c7" (UID: "1d3e9770-476c-47a5-91d7-afffb476b8c7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 22:00:28.279472 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:28.279440 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d3e9770-476c-47a5-91d7-afffb476b8c7-istio-token" pod "1d3e9770-476c-47a5-91d7-afffb476b8c7" (UID: "1d3e9770-476c-47a5-91d7-afffb476b8c7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 03 22:00:28.279867 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:28.279835 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1d3e9770-476c-47a5-91d7-afffb476b8c7-istio-podinfo" pod "1d3e9770-476c-47a5-91d7-afffb476b8c7" (UID: "1d3e9770-476c-47a5-91d7-afffb476b8c7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 03 22:00:28.280382 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:28.280355 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d3e9770-476c-47a5-91d7-afffb476b8c7-kube-api-access-fwrcl" pod "1d3e9770-476c-47a5-91d7-afffb476b8c7" (UID: "1d3e9770-476c-47a5-91d7-afffb476b8c7"). InnerVolumeSpecName "kube-api-access-fwrcl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 03 22:00:28.281364 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:28.281337 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d3e9770-476c-47a5-91d7-afffb476b8c7-istio-envoy" pod "1d3e9770-476c-47a5-91d7-afffb476b8c7" (UID: "1d3e9770-476c-47a5-91d7-afffb476b8c7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 03 22:00:28.376978 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:28.376787 6524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d3e9770-476c-47a5-91d7-afffb476b8c7-workload-socket\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:28.376978 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:28.376829 6524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d3e9770-476c-47a5-91d7-afffb476b8c7-istiod-ca-cert\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:28.376978 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:28.376846 6524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d3e9770-476c-47a5-91d7-afffb476b8c7-istio-data\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:28.376978 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:28.376863 6524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d3e9770-476c-47a5-91d7-afffb476b8c7-istio-podinfo\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:28.376978 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:28.376877 6524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d3e9770-476c-47a5-91d7-afffb476b8c7-istio-envoy\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:28.376978 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:28.376891 6524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d3e9770-476c-47a5-91d7-afffb476b8c7-workload-certs\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:28.376978 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:28.376907 6524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d3e9770-476c-47a5-91d7-afffb476b8c7-istio-token\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:28.376978 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:28.376923 6524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d3e9770-476c-47a5-91d7-afffb476b8c7-credential-socket\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:28.376978 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:28.376938 6524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fwrcl\" (UniqueName: \"kubernetes.io/projected/1d3e9770-476c-47a5-91d7-afffb476b8c7-kube-api-access-fwrcl\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:00:28.804094 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:28.800241 6524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-6czqx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 03 22:00:28.804094 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:28.800330 6524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6czqx" podUID="3379b31a-b827-4511-b8c8-326121b22989" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 03 22:00:28.804094 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:28.803223 6524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-knxdd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 03 22:00:28.804094 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:28.803277 6524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-knxdd" podUID="09c7b2df-6149-4b8a-b8ed-c416fc097757" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 03 22:00:28.828145 ip-10-0-137-7 kubenswrapper[6524]: E1103 22:00:28.827151 6524 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d3e9770_476c_47a5_91d7_afffb476b8c7.slice/crio-9dcf0a782c1b39a9474c1cada5a18f538460be400910cae40f265cf020291245\": RecentStats: unable to find data in memory cache]" Nov 03 22:00:29.149200 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:29.149168 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" Nov 03 22:00:29.149620 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:29.149177 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd" event={"ID":"1d3e9770-476c-47a5-91d7-afffb476b8c7","Type":"ContainerDied","Data":"9dcf0a782c1b39a9474c1cada5a18f538460be400910cae40f265cf020291245"} Nov 03 22:00:29.149620 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:29.149304 6524 scope.go:122] "RemoveContainer" containerID="9c2ab59138d7016737610ce46a8b1def9de74b3e058362dcacd9fab67f6178a9" Nov 03 22:00:29.169427 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:29.169397 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd"] Nov 03 22:00:29.174222 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:29.174193 6524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-53a2cb-openshift-default-7fc9bd965c-zqczd"] Nov 03 22:00:29.712781 ip-10-0-137-7 kubenswrapper[6524]: E1103 22:00:29.712737 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 22:00:30.766675 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:30.766628 6524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1d3e9770-476c-47a5-91d7-afffb476b8c7" path="/var/lib/kubelet/pods/1d3e9770-476c-47a5-91d7-afffb476b8c7/volumes" Nov 03 22:00:54.840432 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:54.840406 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x26mj_76122905-15b4-4f4e-97b9-9f73a7675e36/ovn-acl-logging/0.log" Nov 03 22:00:54.844344 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:00:54.844323 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x26mj_76122905-15b4-4f4e-97b9-9f73a7675e36/ovn-acl-logging/0.log" Nov 03 22:00:59.707600 ip-10-0-137-7 kubenswrapper[6524]: E1103 22:00:59.707560 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 22:01:29.717019 ip-10-0-137-7 kubenswrapper[6524]: E1103 22:01:29.716972 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 22:01:59.707463 ip-10-0-137-7 kubenswrapper[6524]: E1103 22:01:59.707412 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 22:02:29.707758 ip-10-0-137-7 kubenswrapper[6524]: E1103 22:02:29.707720 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 22:02:59.706686 ip-10-0-137-7 kubenswrapper[6524]: E1103 22:02:59.706632 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 22:03:29.707864 ip-10-0-137-7 kubenswrapper[6524]: E1103 22:03:29.707825 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 22:03:59.708366 ip-10-0-137-7 kubenswrapper[6524]: E1103 22:03:59.708316 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 22:04:04.021934 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:04:04.021902 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-61f394fa-s7wth"] Nov 03 22:04:04.024824 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:04:04.024783 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d5888686-q6tdc"] Nov 03 22:04:04.027012 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:04:04.026980 6524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-61f394fa-s7wth"] Nov 03 22:04:04.029689 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:04:04.029665 6524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d5888686-q6tdc"] Nov 03 22:04:04.766993 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:04:04.766958 6524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1eabd95e-9b87-42d7-a442-e7cef3a2c73f" path="/var/lib/kubelet/pods/1eabd95e-9b87-42d7-a442-e7cef3a2c73f/volumes" Nov 03 22:04:04.767367 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:04:04.767353 6524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5b3c5212-f9b9-4845-aab7-a8aac4eacc87" path="/var/lib/kubelet/pods/5b3c5212-f9b9-4845-aab7-a8aac4eacc87/volumes" Nov 03 22:04:19.052318 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:04:19.052209 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-50e65b6f-k2hvk"] Nov 03 22:04:19.054948 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:04:19.054914 6524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-50e65b6f-k2hvk"] Nov 03 22:04:20.766073 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:04:20.766035 6524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4e9aba93-9243-44df-94e7-33ca432c011b" path="/var/lib/kubelet/pods/4e9aba93-9243-44df-94e7-33ca432c011b/volumes" Nov 03 22:04:29.708590 ip-10-0-137-7 kubenswrapper[6524]: E1103 22:04:29.708552 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 22:04:33.024904 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:04:33.024868 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1dcacc9b-57xf9"] Nov 03 22:04:33.027449 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:04:33.027421 6524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1dcacc9b-57xf9"] Nov 03 22:04:34.766977 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:04:34.766939 6524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="86b62a38-13e6-432c-b288-4668c256205e" path="/var/lib/kubelet/pods/86b62a38-13e6-432c-b288-4668c256205e/volumes" Nov 03 22:04:37.022248 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:04:37.022178 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e85a4dd4-zn26z"] Nov 03 22:04:37.027906 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:04:37.027877 6524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e85a4dd4-zn26z"] Nov 03 22:04:38.772817 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:04:38.772781 6524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fdad2bdf-ab1d-4800-9557-3e9b88953a80" path="/var/lib/kubelet/pods/fdad2bdf-ab1d-4800-9557-3e9b88953a80/volumes" Nov 03 22:04:39.022307 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:04:39.022274 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5c184d21-bd7n9"] Nov 03 22:04:39.027373 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:04:39.027302 6524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5c184d21-bd7n9"] Nov 03 22:04:40.767296 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:04:40.767260 6524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="45474bce-683e-4cee-9b5b-0356ed6849f2" path="/var/lib/kubelet/pods/45474bce-683e-4cee-9b5b-0356ed6849f2/volumes" Nov 03 22:04:56.034322 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:04:56.034281 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-97163ab9-7t5ft"] Nov 03 22:04:56.037949 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:04:56.037911 6524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-97163ab9-7t5ft"] Nov 03 22:04:56.767627 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:04:56.767594 6524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5ff565e0-cada-4946-b535-f7cf03d8811e" path="/var/lib/kubelet/pods/5ff565e0-cada-4946-b535-f7cf03d8811e/volumes" Nov 03 22:04:59.707431 ip-10-0-137-7 kubenswrapper[6524]: E1103 22:04:59.707381 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 22:05:05.022921 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:05:05.022881 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-fc2d2e23-p8dlt"] Nov 03 22:05:05.024988 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:05:05.024958 6524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-fc2d2e23-p8dlt"] Nov 03 22:05:06.772298 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:05:06.772258 6524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b25782b8-1123-4fb1-9d07-1f121843c479" path="/var/lib/kubelet/pods/b25782b8-1123-4fb1-9d07-1f121843c479/volumes" Nov 03 22:05:29.708365 ip-10-0-137-7 kubenswrapper[6524]: E1103 22:05:29.708321 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 22:05:54.869780 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:05:54.869747 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x26mj_76122905-15b4-4f4e-97b9-9f73a7675e36/ovn-acl-logging/0.log" Nov 03 22:05:54.874742 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:05:54.874711 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x26mj_76122905-15b4-4f4e-97b9-9f73a7675e36/ovn-acl-logging/0.log" Nov 03 22:05:54.982479 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:05:54.982450 6524 scope.go:122] "RemoveContainer" containerID="1493f3a9441a1eb789d26f04bcd0c8c9f45db5e08c621205781b1b3301136493" Nov 03 22:05:54.990893 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:05:54.990874 6524 scope.go:122] "RemoveContainer" containerID="17157d311048333f1f132b4f014d7f67cbe56fd3bc74a6d4752fd1ab32364717" Nov 03 22:05:55.001632 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:05:55.001606 6524 scope.go:122] "RemoveContainer" containerID="d274abdfe88a42bab5147b02de76c56d3d964430a887971f1ab1365436e955aa" Nov 03 22:05:55.009263 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:05:55.009236 6524 scope.go:122] "RemoveContainer" containerID="e5d9cd564d3bf6b9ebc0bc35ac55182b0770aa8126790255528520a4b25eb550" Nov 03 22:05:55.016106 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:05:55.016085 6524 scope.go:122] "RemoveContainer" containerID="f16ae35446373d360cba236b37ce3fcdf3a3a0bcdd78880b35a7d118161eb0f8" Nov 03 22:05:55.023172 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:05:55.023154 6524 scope.go:122] "RemoveContainer" containerID="7945b79fc07623b834f9549b92ffb9077c6db865b73179e2fcdf31c8723fa1a2" Nov 03 22:05:55.030515 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:05:55.030495 6524 scope.go:122] "RemoveContainer" containerID="1c4c0864c88a507bba31db68196125ed7a9f3476b39e12b3df7f686ccb13818c" Nov 03 22:05:55.037649 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:05:55.037629 6524 scope.go:122] "RemoveContainer" containerID="9d2e643605388a091623a3934da3cfb02f64d9dcee2ca89210ab6ee2b23e4edf" Nov 03 22:05:55.044804 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:05:55.044785 6524 scope.go:122] "RemoveContainer" containerID="8c0edac6b21a3e13967c8bd34431b0bf053b30adcf156f8d45eb7f976f84935f" Nov 03 22:05:59.708314 ip-10-0-137-7 kubenswrapper[6524]: E1103 22:05:59.708276 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 22:06:29.707761 ip-10-0-137-7 kubenswrapper[6524]: E1103 22:06:29.707719 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 22:06:55.087549 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:06:55.087517 6524 scope.go:122] "RemoveContainer" containerID="22f74bb07d5e4ce1c7b8cbde113ee4af59c8b04eadfa27d9390b45e6ffb727e1" Nov 03 22:06:59.707358 ip-10-0-137-7 kubenswrapper[6524]: E1103 22:06:59.707315 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 22:07:29.707360 ip-10-0-137-7 kubenswrapper[6524]: E1103 22:07:29.707318 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 22:07:59.706788 ip-10-0-137-7 kubenswrapper[6524]: E1103 22:07:59.706675 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 22:08:29.708105 ip-10-0-137-7 kubenswrapper[6524]: E1103 22:08:29.708063 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 22:08:59.710544 ip-10-0-137-7 kubenswrapper[6524]: E1103 22:08:59.710498 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 22:09:29.707278 ip-10-0-137-7 kubenswrapper[6524]: E1103 22:09:29.707236 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 22:09:59.706294 ip-10-0-137-7 kubenswrapper[6524]: E1103 22:09:59.706254 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 22:10:29.709392 ip-10-0-137-7 kubenswrapper[6524]: E1103 22:10:29.709351 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 22:10:54.895585 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:10:54.895555 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x26mj_76122905-15b4-4f4e-97b9-9f73a7675e36/ovn-acl-logging/0.log" Nov 03 22:10:54.899588 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:10:54.899564 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x26mj_76122905-15b4-4f4e-97b9-9f73a7675e36/ovn-acl-logging/0.log" Nov 03 22:10:58.137399 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:10:58.137364 6524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-6wdkq"] Nov 03 22:10:58.137792 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:10:58.137704 6524 cpu_manager.go:414] "containerMap: removing container" podUID="587e7995-5cb6-4f61-9f3e-a3fe113a39ad" containerName="cleanup" Nov 03 22:10:58.137792 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:10:58.137718 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="587e7995-5cb6-4f61-9f3e-a3fe113a39ad" containerName="cleanup" Nov 03 22:10:58.137792 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:10:58.137734 6524 cpu_manager.go:414] "containerMap: removing container" podUID="a9593fbc-9af2-4c1f-a986-3dc48dab11a7" containerName="istio-proxy" Nov 03 22:10:58.137792 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:10:58.137739 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a9593fbc-9af2-4c1f-a986-3dc48dab11a7" containerName="istio-proxy" Nov 03 22:10:58.137792 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:10:58.137751 6524 cpu_manager.go:414] "containerMap: removing container" podUID="8462643c-89e1-4ee3-a413-44160da8e03d" containerName="istio-proxy" Nov 03 22:10:58.137792 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:10:58.137756 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8462643c-89e1-4ee3-a413-44160da8e03d" containerName="istio-proxy" Nov 03 22:10:58.137792 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:10:58.137772 6524 cpu_manager.go:414] "containerMap: removing container" podUID="1d3e9770-476c-47a5-91d7-afffb476b8c7" containerName="istio-proxy" Nov 03 22:10:58.137792 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:10:58.137776 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1d3e9770-476c-47a5-91d7-afffb476b8c7" containerName="istio-proxy" Nov 03 22:10:58.137792 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:10:58.137786 6524 cpu_manager.go:414] "containerMap: removing container" podUID="b25782b8-1123-4fb1-9d07-1f121843c479" containerName="revoke-keys" Nov 03 22:10:58.137792 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:10:58.137792 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b25782b8-1123-4fb1-9d07-1f121843c479" containerName="revoke-keys" Nov 03 22:10:58.138268 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:10:58.137849 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="587e7995-5cb6-4f61-9f3e-a3fe113a39ad" containerName="cleanup" Nov 03 22:10:58.138268 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:10:58.137856 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="a9593fbc-9af2-4c1f-a986-3dc48dab11a7" containerName="istio-proxy" Nov 03 22:10:58.138268 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:10:58.137863 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="8462643c-89e1-4ee3-a413-44160da8e03d" containerName="istio-proxy" Nov 03 22:10:58.138268 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:10:58.137872 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="587e7995-5cb6-4f61-9f3e-a3fe113a39ad" containerName="cleanup" Nov 03 22:10:58.138268 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:10:58.137881 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="1d3e9770-476c-47a5-91d7-afffb476b8c7" containerName="istio-proxy" Nov 03 22:10:58.138268 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:10:58.137886 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="b25782b8-1123-4fb1-9d07-1f121843c479" containerName="revoke-keys" Nov 03 22:10:58.141303 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:10:58.141281 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-6wdkq" Nov 03 22:10:58.148962 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:10:58.148540 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Nov 03 22:10:58.148962 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:10:58.148565 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Nov 03 22:10:58.148962 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:10:58.148652 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-dg5g9\"" Nov 03 22:10:58.154090 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:10:58.154056 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-6wdkq"] Nov 03 22:10:58.272248 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:10:58.272213 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79-maas-api-tls\") pod \"maas-api-5c48877c75-6wdkq\" (UID: \"a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-6wdkq" Nov 03 22:10:58.272439 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:10:58.272275 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79-openshift-service-ca\") pod \"maas-api-5c48877c75-6wdkq\" (UID: \"a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-6wdkq" Nov 03 22:10:58.272439 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:10:58.272410 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79-metrics-certs\") pod \"maas-api-5c48877c75-6wdkq\" (UID: \"a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-6wdkq" Nov 03 22:10:58.272524 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:10:58.272449 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zp7qz\" (UniqueName: \"kubernetes.io/projected/a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79-kube-api-access-zp7qz\") pod \"maas-api-5c48877c75-6wdkq\" (UID: \"a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-6wdkq" Nov 03 22:10:58.373265 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:10:58.373225 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79-metrics-certs\") pod \"maas-api-5c48877c75-6wdkq\" (UID: \"a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-6wdkq" Nov 03 22:10:58.373265 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:10:58.373271 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zp7qz\" (UniqueName: \"kubernetes.io/projected/a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79-kube-api-access-zp7qz\") pod \"maas-api-5c48877c75-6wdkq\" (UID: \"a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-6wdkq" Nov 03 22:10:58.373542 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:10:58.373324 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79-maas-api-tls\") pod \"maas-api-5c48877c75-6wdkq\" (UID: \"a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-6wdkq" Nov 03 22:10:58.373542 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:10:58.373351 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79-openshift-service-ca\") pod \"maas-api-5c48877c75-6wdkq\" (UID: \"a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-6wdkq" Nov 03 22:10:58.373975 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:10:58.373952 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79-openshift-service-ca\") pod \"maas-api-5c48877c75-6wdkq\" (UID: \"a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-6wdkq" Nov 03 22:10:58.375717 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:10:58.375694 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79-metrics-certs\") pod \"maas-api-5c48877c75-6wdkq\" (UID: \"a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-6wdkq" Nov 03 22:10:58.375803 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:10:58.375763 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79-maas-api-tls\") pod \"maas-api-5c48877c75-6wdkq\" (UID: \"a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-6wdkq" Nov 03 22:10:58.387907 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:10:58.387845 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zp7qz\" (UniqueName: \"kubernetes.io/projected/a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79-kube-api-access-zp7qz\") pod \"maas-api-5c48877c75-6wdkq\" (UID: \"a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-6wdkq" Nov 03 22:10:58.457902 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:10:58.457861 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-6wdkq" Nov 03 22:10:58.731365 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:10:58.731290 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-6wdkq"] Nov 03 22:10:58.734288 ip-10-0-137-7 kubenswrapper[6524]: W1103 22:10:58.734258 6524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda659ed4e_1f52_4b6c_a2a6_1e15cd4b9b79.slice/crio-5fc8069ca9cb83fbf14744ed06c339b12a86fa79ca05cdcc832be01835605b67 WatchSource:0}: Error finding container 5fc8069ca9cb83fbf14744ed06c339b12a86fa79ca05cdcc832be01835605b67: Status 404 returned error can't find the container with id 5fc8069ca9cb83fbf14744ed06c339b12a86fa79ca05cdcc832be01835605b67 Nov 03 22:10:58.735540 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:10:58.735522 6524 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 03 22:10:59.326282 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:10:59.326242 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-6wdkq" event={"ID":"a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79","Type":"ContainerStarted","Data":"a301296ed2bee6d57e08e10685a34baeceb7240806f33ca8a13cc37791cd0624"} Nov 03 22:10:59.326700 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:10:59.326311 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-6wdkq" event={"ID":"a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79","Type":"ContainerStarted","Data":"5fc8069ca9cb83fbf14744ed06c339b12a86fa79ca05cdcc832be01835605b67"} Nov 03 22:10:59.326700 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:10:59.326342 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5c48877c75-6wdkq" Nov 03 22:10:59.346820 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:10:59.346762 6524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5c48877c75-6wdkq" podStartSLOduration=0.864836623 podStartE2EDuration="1.346748554s" totalImagesPullingTime="481.911931ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:10:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:10:59.34559503 +0000 UTC m=+1504.697921835" watchObservedRunningTime="2026-11-03 22:10:59.346748554 +0000 UTC m=+1504.699075359" Nov 03 22:10:59.707459 ip-10-0-137-7 kubenswrapper[6524]: E1103 22:10:59.707422 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 22:11:05.337344 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:05.337315 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5c48877c75-6wdkq" Nov 03 22:11:06.271819 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:06.271782 6524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-595cc648c-srwl4"] Nov 03 22:11:06.272188 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:06.272174 6524 cpu_manager.go:414] "containerMap: removing container" podUID="587e7995-5cb6-4f61-9f3e-a3fe113a39ad" containerName="cleanup" Nov 03 22:11:06.272245 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:06.272189 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="587e7995-5cb6-4f61-9f3e-a3fe113a39ad" containerName="cleanup" Nov 03 22:11:06.275359 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:06.275342 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-595cc648c-srwl4" Nov 03 22:11:06.286870 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:06.286839 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-595cc648c-srwl4"] Nov 03 22:11:06.365323 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:06.365287 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae3a1939-13ef-43dc-ab7b-fa82ceffa64b-maas-api-tls\") pod \"maas-api-595cc648c-srwl4\" (UID: \"ae3a1939-13ef-43dc-ab7b-fa82ceffa64b\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-srwl4" Nov 03 22:11:06.365726 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:06.365329 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae3a1939-13ef-43dc-ab7b-fa82ceffa64b-metrics-certs\") pod \"maas-api-595cc648c-srwl4\" (UID: \"ae3a1939-13ef-43dc-ab7b-fa82ceffa64b\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-srwl4" Nov 03 22:11:06.365726 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:06.365361 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4c9h\" (UniqueName: \"kubernetes.io/projected/ae3a1939-13ef-43dc-ab7b-fa82ceffa64b-kube-api-access-t4c9h\") pod \"maas-api-595cc648c-srwl4\" (UID: \"ae3a1939-13ef-43dc-ab7b-fa82ceffa64b\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-srwl4" Nov 03 22:11:06.365726 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:06.365477 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae3a1939-13ef-43dc-ab7b-fa82ceffa64b-openshift-service-ca\") pod \"maas-api-595cc648c-srwl4\" (UID: \"ae3a1939-13ef-43dc-ab7b-fa82ceffa64b\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-srwl4" Nov 03 22:11:06.466316 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:06.466268 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae3a1939-13ef-43dc-ab7b-fa82ceffa64b-maas-api-tls\") pod \"maas-api-595cc648c-srwl4\" (UID: \"ae3a1939-13ef-43dc-ab7b-fa82ceffa64b\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-srwl4" Nov 03 22:11:06.466535 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:06.466357 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae3a1939-13ef-43dc-ab7b-fa82ceffa64b-metrics-certs\") pod \"maas-api-595cc648c-srwl4\" (UID: \"ae3a1939-13ef-43dc-ab7b-fa82ceffa64b\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-srwl4" Nov 03 22:11:06.466535 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:06.466419 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4c9h\" (UniqueName: \"kubernetes.io/projected/ae3a1939-13ef-43dc-ab7b-fa82ceffa64b-kube-api-access-t4c9h\") pod \"maas-api-595cc648c-srwl4\" (UID: \"ae3a1939-13ef-43dc-ab7b-fa82ceffa64b\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-srwl4" Nov 03 22:11:06.466535 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:06.466471 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae3a1939-13ef-43dc-ab7b-fa82ceffa64b-openshift-service-ca\") pod \"maas-api-595cc648c-srwl4\" (UID: \"ae3a1939-13ef-43dc-ab7b-fa82ceffa64b\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-srwl4" Nov 03 22:11:06.467063 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:06.467046 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae3a1939-13ef-43dc-ab7b-fa82ceffa64b-openshift-service-ca\") pod \"maas-api-595cc648c-srwl4\" (UID: \"ae3a1939-13ef-43dc-ab7b-fa82ceffa64b\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-srwl4" Nov 03 22:11:06.468718 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:06.468695 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae3a1939-13ef-43dc-ab7b-fa82ceffa64b-metrics-certs\") pod \"maas-api-595cc648c-srwl4\" (UID: \"ae3a1939-13ef-43dc-ab7b-fa82ceffa64b\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-srwl4" Nov 03 22:11:06.468813 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:06.468731 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae3a1939-13ef-43dc-ab7b-fa82ceffa64b-maas-api-tls\") pod \"maas-api-595cc648c-srwl4\" (UID: \"ae3a1939-13ef-43dc-ab7b-fa82ceffa64b\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-srwl4" Nov 03 22:11:06.479265 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:06.479241 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4c9h\" (UniqueName: \"kubernetes.io/projected/ae3a1939-13ef-43dc-ab7b-fa82ceffa64b-kube-api-access-t4c9h\") pod \"maas-api-595cc648c-srwl4\" (UID: \"ae3a1939-13ef-43dc-ab7b-fa82ceffa64b\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-srwl4" Nov 03 22:11:06.589473 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:06.589437 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-595cc648c-srwl4" Nov 03 22:11:06.870346 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:06.870246 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-595cc648c-srwl4"] Nov 03 22:11:06.875698 ip-10-0-137-7 kubenswrapper[6524]: W1103 22:11:06.875654 6524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae3a1939_13ef_43dc_ab7b_fa82ceffa64b.slice/crio-be4e23b97a4dc865e4d60e902d9aa0b0d97d36da6468cf660f2b37a5229aa635 WatchSource:0}: Error finding container be4e23b97a4dc865e4d60e902d9aa0b0d97d36da6468cf660f2b37a5229aa635: Status 404 returned error can't find the container with id be4e23b97a4dc865e4d60e902d9aa0b0d97d36da6468cf660f2b37a5229aa635 Nov 03 22:11:07.369502 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:07.369463 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-595cc648c-srwl4" event={"ID":"ae3a1939-13ef-43dc-ab7b-fa82ceffa64b","Type":"ContainerStarted","Data":"be4e23b97a4dc865e4d60e902d9aa0b0d97d36da6468cf660f2b37a5229aa635"} Nov 03 22:11:08.375708 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:08.375675 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-595cc648c-srwl4" event={"ID":"ae3a1939-13ef-43dc-ab7b-fa82ceffa64b","Type":"ContainerStarted","Data":"28551315a59d62de584726e4b3459e913c596b22768a3f4895f6dab7bd3dbaea"} Nov 03 22:11:08.376102 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:08.375789 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-595cc648c-srwl4" Nov 03 22:11:08.405855 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:08.405773 6524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-595cc648c-srwl4" podStartSLOduration=1.949211332 podStartE2EDuration="2.405755558s" totalImagesPullingTime="456.544226ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 22:11:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:11:08.39863276 +0000 UTC m=+1513.750959565" watchObservedRunningTime="2026-11-03 22:11:08.405755558 +0000 UTC m=+1513.758082362" Nov 03 22:11:14.385233 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:14.385203 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-595cc648c-srwl4" Nov 03 22:11:14.436003 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:14.435963 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-6wdkq"] Nov 03 22:11:14.436206 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:14.436171 6524 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-5c48877c75-6wdkq" podUID="a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79" containerName="maas-api" containerID="cri-o://a301296ed2bee6d57e08e10685a34baeceb7240806f33ca8a13cc37791cd0624" gracePeriod=30 Nov 03 22:11:15.332812 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:15.332773 6524 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-5c48877c75-6wdkq" podUID="a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.52:8443/health\": dial tcp 10.133.0.52:8443: connect: connection refused" Nov 03 22:11:15.686335 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:15.686306 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-6wdkq" Nov 03 22:11:15.715503 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:15.715460 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79-maas-api-tls\") pod \"a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79\" (UID: \"a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79\") " Nov 03 22:11:15.715711 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:15.715526 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79-metrics-certs\") pod \"a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79\" (UID: \"a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79\") " Nov 03 22:11:15.715711 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:15.715582 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79-kube-api-access-zp7qz\" (UniqueName: \"kubernetes.io/projected/a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79-kube-api-access-zp7qz\") pod \"a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79\" (UID: \"a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79\") " Nov 03 22:11:15.715711 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:15.715612 6524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79-openshift-service-ca\") pod \"a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79\" (UID: \"a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79\") " Nov 03 22:11:15.716076 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:15.716050 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79-openshift-service-ca" pod "a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79" (UID: "a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 03 22:11:15.718218 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:15.718178 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79-maas-api-tls" pod "a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79" (UID: "a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 03 22:11:15.718363 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:15.718336 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79-kube-api-access-zp7qz" pod "a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79" (UID: "a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79"). InnerVolumeSpecName "kube-api-access-zp7qz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 03 22:11:15.718651 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:15.718632 6524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79-metrics-certs" pod "a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79" (UID: "a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 03 22:11:15.816521 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:15.816482 6524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79-maas-api-tls\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:11:15.816521 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:15.816518 6524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79-metrics-certs\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:11:15.816521 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:15.816533 6524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zp7qz\" (UniqueName: \"kubernetes.io/projected/a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79-kube-api-access-zp7qz\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:11:15.816818 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:15.816542 6524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79-openshift-service-ca\") on node \"ip-10-0-137-7.us-east-2.compute.internal\" DevicePath \"\"" Nov 03 22:11:16.410270 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:16.410230 6524 generic.go:356] "Generic (PLEG): container finished" podID="a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79" containerID="a301296ed2bee6d57e08e10685a34baeceb7240806f33ca8a13cc37791cd0624" exitCode=0 Nov 03 22:11:16.410453 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:16.410283 6524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-6wdkq" Nov 03 22:11:16.410453 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:16.410312 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-6wdkq" event={"ID":"a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79","Type":"ContainerDied","Data":"a301296ed2bee6d57e08e10685a34baeceb7240806f33ca8a13cc37791cd0624"} Nov 03 22:11:16.410453 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:16.410365 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-6wdkq" event={"ID":"a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79","Type":"ContainerDied","Data":"5fc8069ca9cb83fbf14744ed06c339b12a86fa79ca05cdcc832be01835605b67"} Nov 03 22:11:16.410453 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:16.410387 6524 scope.go:122] "RemoveContainer" containerID="a301296ed2bee6d57e08e10685a34baeceb7240806f33ca8a13cc37791cd0624" Nov 03 22:11:16.419361 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:16.419298 6524 scope.go:122] "RemoveContainer" containerID="a301296ed2bee6d57e08e10685a34baeceb7240806f33ca8a13cc37791cd0624" Nov 03 22:11:16.419685 ip-10-0-137-7 kubenswrapper[6524]: E1103 22:11:16.419654 6524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a301296ed2bee6d57e08e10685a34baeceb7240806f33ca8a13cc37791cd0624\": container with ID starting with a301296ed2bee6d57e08e10685a34baeceb7240806f33ca8a13cc37791cd0624 not found: ID does not exist" containerID="a301296ed2bee6d57e08e10685a34baeceb7240806f33ca8a13cc37791cd0624" Nov 03 22:11:16.419766 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:16.419687 6524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a301296ed2bee6d57e08e10685a34baeceb7240806f33ca8a13cc37791cd0624"} err="failed to get container status \"a301296ed2bee6d57e08e10685a34baeceb7240806f33ca8a13cc37791cd0624\": rpc error: code = NotFound desc = could not find container \"a301296ed2bee6d57e08e10685a34baeceb7240806f33ca8a13cc37791cd0624\": container with ID starting with a301296ed2bee6d57e08e10685a34baeceb7240806f33ca8a13cc37791cd0624 not found: ID does not exist" Nov 03 22:11:16.431830 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:16.431805 6524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-6wdkq"] Nov 03 22:11:16.435845 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:16.435821 6524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-6wdkq"] Nov 03 22:11:16.767386 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:16.767307 6524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79" path="/var/lib/kubelet/pods/a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79/volumes" Nov 03 22:11:29.710059 ip-10-0-137-7 kubenswrapper[6524]: E1103 22:11:29.710012 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 22:11:59.398061 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:59.398033 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-5cfb77bf8c-hg589_38f1f654-c620-48a0-a6dc-748bbc212279/manager/0.log" Nov 03 22:11:59.597432 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:11:59.597395 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-k99zx_8559b8d0-420f-4b4c-a430-bb54f19c0fa6/manager/0.log" Nov 03 22:11:59.707034 ip-10-0-137-7 kubenswrapper[6524]: E1103 22:11:59.706952 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 22:12:00.654191 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:12:00.654164 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-77f6458f94-jpmrh_0f1eb659-a77c-4638-8326-6be9dc29e08d/manager/0.log" Nov 03 22:12:00.673519 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:12:00.673493 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-77f6458f94-jpmrh_0f1eb659-a77c-4638-8326-6be9dc29e08d/copy-manifests/0.log" Nov 03 22:12:02.098286 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:12:02.098258 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-595cc648c-srwl4_ae3a1939-13ef-43dc-ab7b-fa82ceffa64b/maas-api/0.log" Nov 03 22:12:03.997306 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:12:03.997277 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-vkpbm_4844f9ef-f2e3-4483-ae7a-fcb57a4ded1e/postgres/0.log" Nov 03 22:12:07.595154 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:12:07.595094 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-7vrmd_497ce0ef-46c2-4727-8424-e676f303bde5/discovery/0.log" Nov 03 22:12:08.129036 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:12:08.129007 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-66db998887-2jqqp_a85e1f40-b852-4024-a1eb-0c5798b081e7/istio-proxy/0.log" Nov 03 22:12:08.669798 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:12:08.669769 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-66db998887-6czqx_3379b31a-b827-4511-b8c8-326121b22989/istio-proxy/0.log" Nov 03 22:12:09.602138 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:12:09.602089 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-66db998887-knxdd_09c7b2df-6149-4b8a-b8ed-c416fc097757/istio-proxy/0.log" Nov 03 22:12:22.378831 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:12:22.378790 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-5cfb77bf8c-hg589_38f1f654-c620-48a0-a6dc-748bbc212279/manager/0.log" Nov 03 22:12:22.588693 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:12:22.588661 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-k99zx_8559b8d0-420f-4b4c-a430-bb54f19c0fa6/manager/0.log" Nov 03 22:12:23.629819 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:12:23.629786 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-77f6458f94-jpmrh_0f1eb659-a77c-4638-8326-6be9dc29e08d/copy-manifests/0.log" Nov 03 22:12:23.649781 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:12:23.649750 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-77f6458f94-jpmrh_0f1eb659-a77c-4638-8326-6be9dc29e08d/manager/0.log" Nov 03 22:12:24.825707 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:12:24.825680 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-595cc648c-srwl4_ae3a1939-13ef-43dc-ab7b-fa82ceffa64b/maas-api/0.log" Nov 03 22:12:26.711617 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:12:26.711589 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-vkpbm_4844f9ef-f2e3-4483-ae7a-fcb57a4ded1e/postgres/0.log" Nov 03 22:12:29.667235 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:12:29.667194 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-7vrmd_497ce0ef-46c2-4727-8424-e676f303bde5/discovery/0.log" Nov 03 22:12:29.707631 ip-10-0-137-7 kubenswrapper[6524]: E1103 22:12:29.707587 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 22:12:30.216538 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:12:30.216500 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-66db998887-2jqqp_a85e1f40-b852-4024-a1eb-0c5798b081e7/istio-proxy/0.log" Nov 03 22:12:30.770456 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:12:30.770424 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-66db998887-6czqx_3379b31a-b827-4511-b8c8-326121b22989/istio-proxy/0.log" Nov 03 22:12:31.696260 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:12:31.696228 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-66db998887-knxdd_09c7b2df-6149-4b8a-b8ed-c416fc097757/istio-proxy/0.log" Nov 03 22:12:59.706622 ip-10-0-137-7 kubenswrapper[6524]: E1103 22:12:59.706584 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 22:13:29.707205 ip-10-0-137-7 kubenswrapper[6524]: E1103 22:13:29.707162 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 22:13:59.708090 ip-10-0-137-7 kubenswrapper[6524]: E1103 22:13:59.708046 6524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-285hg" containerName="extractor" Nov 03 22:14:01.272960 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:01.272931 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-jq6hj_94aaa3ac-c7fc-4f87-878e-8ef79127488b/global-pull-secret-syncer/0.log" Nov 03 22:14:01.513047 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:01.513005 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-v98zn_811514f5-baff-4fc1-9e92-28f27de7b0b7/konnectivity-agent/0.log" Nov 03 22:14:01.593200 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:01.593171 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-7.us-east-2.compute.internal_70b8abbfbbbc47301b7da80ca6b44429/haproxy/0.log" Nov 03 22:14:10.422196 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:10.422107 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-77f4f76594-9cnqh_a6cf0029-66be-4d86-ae8e-713d173a113e/monitoring-plugin/0.log" Nov 03 22:14:10.638320 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:10.638294 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jdbs4_bfd8df86-d3d0-4576-afd0-fa742a6a3aad/node-exporter/0.log" Nov 03 22:14:10.663661 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:10.663628 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jdbs4_bfd8df86-d3d0-4576-afd0-fa742a6a3aad/kube-rbac-proxy/0.log" Nov 03 22:14:10.689543 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:10.689469 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jdbs4_bfd8df86-d3d0-4576-afd0-fa742a6a3aad/init-textfile/0.log" Nov 03 22:14:10.815192 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:10.815160 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_327ba0ad-9702-4e8b-bd97-8b6792b03e51/prometheus/0.log" Nov 03 22:14:10.841269 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:10.841239 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_327ba0ad-9702-4e8b-bd97-8b6792b03e51/config-reloader/0.log" Nov 03 22:14:10.867725 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:10.867697 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_327ba0ad-9702-4e8b-bd97-8b6792b03e51/thanos-sidecar/0.log" Nov 03 22:14:10.893626 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:10.893597 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_327ba0ad-9702-4e8b-bd97-8b6792b03e51/kube-rbac-proxy-web/0.log" Nov 03 22:14:10.919635 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:10.919608 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_327ba0ad-9702-4e8b-bd97-8b6792b03e51/kube-rbac-proxy/0.log" Nov 03 22:14:10.952833 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:10.952761 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_327ba0ad-9702-4e8b-bd97-8b6792b03e51/kube-rbac-proxy-thanos/0.log" Nov 03 22:14:10.984331 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:10.984304 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_327ba0ad-9702-4e8b-bd97-8b6792b03e51/init-config-reloader/0.log" Nov 03 22:14:10.993137 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:10.993085 6524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-69rm6/perf-node-gather-daemonset-zqwk4"] Nov 03 22:14:10.993543 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:10.993522 6524 cpu_manager.go:414] "containerMap: removing container" podUID="a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79" containerName="maas-api" Nov 03 22:14:10.993610 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:10.993546 6524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79" containerName="maas-api" Nov 03 22:14:10.993668 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:10.993657 6524 memory_manager.go:365] "RemoveStaleState removing state" podUID="a659ed4e-1f52-4b6c-a2a6-1e15cd4b9b79" containerName="maas-api" Nov 03 22:14:10.996628 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:10.996613 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-69rm6/perf-node-gather-daemonset-zqwk4" Nov 03 22:14:11.007517 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:11.007489 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-69rm6/perf-node-gather-daemonset-zqwk4"] Nov 03 22:14:11.047065 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:11.047028 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcwh2\" (UniqueName: \"kubernetes.io/projected/6a8fe308-7481-4041-93ec-f3b7e077a9c3-kube-api-access-lcwh2\") pod \"perf-node-gather-daemonset-zqwk4\" (UID: \"6a8fe308-7481-4041-93ec-f3b7e077a9c3\") " pod="openshift-must-gather-69rm6/perf-node-gather-daemonset-zqwk4" Nov 03 22:14:11.047287 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:11.047075 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6a8fe308-7481-4041-93ec-f3b7e077a9c3-sys\") pod \"perf-node-gather-daemonset-zqwk4\" (UID: \"6a8fe308-7481-4041-93ec-f3b7e077a9c3\") " pod="openshift-must-gather-69rm6/perf-node-gather-daemonset-zqwk4" Nov 03 22:14:11.047287 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:11.047185 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6a8fe308-7481-4041-93ec-f3b7e077a9c3-podres\") pod \"perf-node-gather-daemonset-zqwk4\" (UID: \"6a8fe308-7481-4041-93ec-f3b7e077a9c3\") " pod="openshift-must-gather-69rm6/perf-node-gather-daemonset-zqwk4" Nov 03 22:14:11.047287 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:11.047263 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6a8fe308-7481-4041-93ec-f3b7e077a9c3-proc\") pod \"perf-node-gather-daemonset-zqwk4\" (UID: \"6a8fe308-7481-4041-93ec-f3b7e077a9c3\") " pod="openshift-must-gather-69rm6/perf-node-gather-daemonset-zqwk4" Nov 03 22:14:11.047428 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:11.047403 6524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6a8fe308-7481-4041-93ec-f3b7e077a9c3-lib-modules\") pod \"perf-node-gather-daemonset-zqwk4\" (UID: \"6a8fe308-7481-4041-93ec-f3b7e077a9c3\") " pod="openshift-must-gather-69rm6/perf-node-gather-daemonset-zqwk4" Nov 03 22:14:11.093281 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:11.093251 6524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-69rm6\"/\"default-dockercfg-wv8q7\"" Nov 03 22:14:11.093674 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:11.093661 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-69rm6\"/\"kube-root-ca.crt\"" Nov 03 22:14:11.093799 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:11.093782 6524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-69rm6\"/\"openshift-service-ca.crt\"" Nov 03 22:14:11.147880 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:11.147845 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6a8fe308-7481-4041-93ec-f3b7e077a9c3-proc\") pod \"perf-node-gather-daemonset-zqwk4\" (UID: \"6a8fe308-7481-4041-93ec-f3b7e077a9c3\") " pod="openshift-must-gather-69rm6/perf-node-gather-daemonset-zqwk4" Nov 03 22:14:11.148080 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:11.147938 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6a8fe308-7481-4041-93ec-f3b7e077a9c3-lib-modules\") pod \"perf-node-gather-daemonset-zqwk4\" (UID: \"6a8fe308-7481-4041-93ec-f3b7e077a9c3\") " pod="openshift-must-gather-69rm6/perf-node-gather-daemonset-zqwk4" Nov 03 22:14:11.148080 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:11.147969 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6a8fe308-7481-4041-93ec-f3b7e077a9c3-proc\") pod \"perf-node-gather-daemonset-zqwk4\" (UID: \"6a8fe308-7481-4041-93ec-f3b7e077a9c3\") " pod="openshift-must-gather-69rm6/perf-node-gather-daemonset-zqwk4" Nov 03 22:14:11.148080 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:11.147986 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcwh2\" (UniqueName: \"kubernetes.io/projected/6a8fe308-7481-4041-93ec-f3b7e077a9c3-kube-api-access-lcwh2\") pod \"perf-node-gather-daemonset-zqwk4\" (UID: \"6a8fe308-7481-4041-93ec-f3b7e077a9c3\") " pod="openshift-must-gather-69rm6/perf-node-gather-daemonset-zqwk4" Nov 03 22:14:11.148080 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:11.148015 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6a8fe308-7481-4041-93ec-f3b7e077a9c3-sys\") pod \"perf-node-gather-daemonset-zqwk4\" (UID: \"6a8fe308-7481-4041-93ec-f3b7e077a9c3\") " pod="openshift-must-gather-69rm6/perf-node-gather-daemonset-zqwk4" Nov 03 22:14:11.148080 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:11.148031 6524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6a8fe308-7481-4041-93ec-f3b7e077a9c3-podres\") pod \"perf-node-gather-daemonset-zqwk4\" (UID: \"6a8fe308-7481-4041-93ec-f3b7e077a9c3\") " pod="openshift-must-gather-69rm6/perf-node-gather-daemonset-zqwk4" Nov 03 22:14:11.148080 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:11.148069 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6a8fe308-7481-4041-93ec-f3b7e077a9c3-lib-modules\") pod \"perf-node-gather-daemonset-zqwk4\" (UID: \"6a8fe308-7481-4041-93ec-f3b7e077a9c3\") " pod="openshift-must-gather-69rm6/perf-node-gather-daemonset-zqwk4" Nov 03 22:14:11.148394 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:11.148116 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6a8fe308-7481-4041-93ec-f3b7e077a9c3-sys\") pod \"perf-node-gather-daemonset-zqwk4\" (UID: \"6a8fe308-7481-4041-93ec-f3b7e077a9c3\") " pod="openshift-must-gather-69rm6/perf-node-gather-daemonset-zqwk4" Nov 03 22:14:11.148394 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:11.148149 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6a8fe308-7481-4041-93ec-f3b7e077a9c3-podres\") pod \"perf-node-gather-daemonset-zqwk4\" (UID: \"6a8fe308-7481-4041-93ec-f3b7e077a9c3\") " pod="openshift-must-gather-69rm6/perf-node-gather-daemonset-zqwk4" Nov 03 22:14:11.160597 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:11.160561 6524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcwh2\" (UniqueName: \"kubernetes.io/projected/6a8fe308-7481-4041-93ec-f3b7e077a9c3-kube-api-access-lcwh2\") pod \"perf-node-gather-daemonset-zqwk4\" (UID: \"6a8fe308-7481-4041-93ec-f3b7e077a9c3\") " pod="openshift-must-gather-69rm6/perf-node-gather-daemonset-zqwk4" Nov 03 22:14:11.222144 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:11.222053 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-54674f497-2g5ct_00c924c1-ac0d-49c8-8610-3b837fbb8277/thanos-query/0.log" Nov 03 22:14:11.247361 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:11.247319 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-54674f497-2g5ct_00c924c1-ac0d-49c8-8610-3b837fbb8277/kube-rbac-proxy-web/0.log" Nov 03 22:14:11.280022 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:11.279994 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-54674f497-2g5ct_00c924c1-ac0d-49c8-8610-3b837fbb8277/kube-rbac-proxy/0.log" Nov 03 22:14:11.305697 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:11.305665 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-54674f497-2g5ct_00c924c1-ac0d-49c8-8610-3b837fbb8277/prom-label-proxy/0.log" Nov 03 22:14:11.309754 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:11.309736 6524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-69rm6/perf-node-gather-daemonset-zqwk4" Nov 03 22:14:11.331344 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:11.331320 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-54674f497-2g5ct_00c924c1-ac0d-49c8-8610-3b837fbb8277/kube-rbac-proxy-rules/0.log" Nov 03 22:14:11.362158 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:11.359789 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-54674f497-2g5ct_00c924c1-ac0d-49c8-8610-3b837fbb8277/kube-rbac-proxy-metrics/0.log" Nov 03 22:14:11.584583 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:11.584548 6524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-69rm6/perf-node-gather-daemonset-zqwk4"] Nov 03 22:14:11.587847 ip-10-0-137-7 kubenswrapper[6524]: W1103 22:14:11.587820 6524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6a8fe308_7481_4041_93ec_f3b7e077a9c3.slice/crio-ab83171c565709d1196e4dbe4fc6339eaed92c3ed489b3f0a6287e18c250f373 WatchSource:0}: Error finding container ab83171c565709d1196e4dbe4fc6339eaed92c3ed489b3f0a6287e18c250f373: Status 404 returned error can't find the container with id ab83171c565709d1196e4dbe4fc6339eaed92c3ed489b3f0a6287e18c250f373 Nov 03 22:14:12.154871 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:12.154832 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-69rm6/perf-node-gather-daemonset-zqwk4" event={"ID":"6a8fe308-7481-4041-93ec-f3b7e077a9c3","Type":"ContainerStarted","Data":"e6726f736012136ad69318e09be3e78ca76ddbeb7a92f88c3a0f40f51d704736"} Nov 03 22:14:12.155074 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:12.154883 6524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-69rm6/perf-node-gather-daemonset-zqwk4" event={"ID":"6a8fe308-7481-4041-93ec-f3b7e077a9c3","Type":"ContainerStarted","Data":"ab83171c565709d1196e4dbe4fc6339eaed92c3ed489b3f0a6287e18c250f373"} Nov 03 22:14:12.155074 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:12.154974 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-69rm6/perf-node-gather-daemonset-zqwk4" Nov 03 22:14:12.175837 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:12.175784 6524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-69rm6/perf-node-gather-daemonset-zqwk4" podStartSLOduration=2.175770703 podStartE2EDuration="2.175770703s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 22:14:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 22:14:12.174535604 +0000 UTC m=+1697.526862409" watchObservedRunningTime="2026-11-03 22:14:12.175770703 +0000 UTC m=+1697.528097510" Nov 03 22:14:15.566070 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:15.566041 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-x2r5s_5921d9cf-54a7-43bd-b697-c5ccf9f39135/dns/0.log" Nov 03 22:14:15.593492 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:15.593458 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-x2r5s_5921d9cf-54a7-43bd-b697-c5ccf9f39135/kube-rbac-proxy/0.log" Nov 03 22:14:15.652243 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:15.652200 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9pctw_9f5c183e-d58a-4621-940c-04a7ae0bafc9/dns-node-resolver/0.log" Nov 03 22:14:16.414457 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:16.414424 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nz4zw_a440580e-b0bd-49e2-a33d-576118e049ae/node-ca/0.log" Nov 03 22:14:18.163864 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:18.163811 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-7vrmd_497ce0ef-46c2-4727-8424-e676f303bde5/discovery/0.log" Nov 03 22:14:18.169525 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:18.169498 6524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-69rm6/perf-node-gather-daemonset-zqwk4" Nov 03 22:14:18.260388 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:18.260355 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-66db998887-2jqqp_a85e1f40-b852-4024-a1eb-0c5798b081e7/istio-proxy/0.log" Nov 03 22:14:18.356772 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:18.356739 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-66db998887-6czqx_3379b31a-b827-4511-b8c8-326121b22989/istio-proxy/0.log" Nov 03 22:14:18.522044 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:18.521967 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-66db998887-knxdd_09c7b2df-6149-4b8a-b8ed-c416fc097757/istio-proxy/0.log" Nov 03 22:14:19.613260 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:19.613227 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-77wdg_32fd1d86-45ed-4a7d-894e-3990980bc237/serve-healthcheck-canary/0.log" Nov 03 22:14:20.286386 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:20.286355 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-285hg_e652c3ec-d5ac-43f9-ade5-d3da32041cee/kube-rbac-proxy/0.log" Nov 03 22:14:20.312133 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:20.312106 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-285hg_e652c3ec-d5ac-43f9-ade5-d3da32041cee/exporter/0.log" Nov 03 22:14:20.338517 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:20.338484 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-285hg_e652c3ec-d5ac-43f9-ade5-d3da32041cee/extractor/0.log" Nov 03 22:14:22.169735 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:22.169699 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-7vrmd_497ce0ef-46c2-4727-8424-e676f303bde5/discovery/0.log" Nov 03 22:14:22.219045 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:22.219019 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-66db998887-2jqqp_a85e1f40-b852-4024-a1eb-0c5798b081e7/istio-proxy/0.log" Nov 03 22:14:22.272647 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:22.272621 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-66db998887-6czqx_3379b31a-b827-4511-b8c8-326121b22989/istio-proxy/0.log" Nov 03 22:14:22.369712 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:22.369683 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-66db998887-knxdd_09c7b2df-6149-4b8a-b8ed-c416fc097757/istio-proxy/0.log" Nov 03 22:14:23.530061 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:23.530027 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-5cfb77bf8c-hg589_38f1f654-c620-48a0-a6dc-748bbc212279/manager/0.log" Nov 03 22:14:23.680908 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:23.680876 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-k99zx_8559b8d0-420f-4b4c-a430-bb54f19c0fa6/manager/0.log" Nov 03 22:14:23.999713 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:23.999687 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-77f6458f94-jpmrh_0f1eb659-a77c-4638-8326-6be9dc29e08d/manager/0.log" Nov 03 22:14:24.025604 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:24.025573 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-77f6458f94-jpmrh_0f1eb659-a77c-4638-8326-6be9dc29e08d/copy-manifests/0.log" Nov 03 22:14:24.536357 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:24.536318 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-7vrmd_497ce0ef-46c2-4727-8424-e676f303bde5/discovery/0.log" Nov 03 22:14:24.601716 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:24.601683 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-66db998887-2jqqp_a85e1f40-b852-4024-a1eb-0c5798b081e7/istio-proxy/0.log" Nov 03 22:14:24.652507 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:24.652477 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-66db998887-6czqx_3379b31a-b827-4511-b8c8-326121b22989/istio-proxy/0.log" Nov 03 22:14:24.744611 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:24.744582 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-66db998887-knxdd_09c7b2df-6149-4b8a-b8ed-c416fc097757/istio-proxy/0.log" Nov 03 22:14:25.589597 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:25.589570 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6c6f488f88-7rl88_f614ff28-8d30-47e9-a78f-cd2078f2d35c/manager/0.log" Nov 03 22:14:25.647687 ip-10-0-137-7 kubenswrapper[6524]: I1103 22:14:25.647657 6524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-8465994787-6kp69_5806ca3d-887b-4b37-a0fe-31303425d395/openshift-lws-operator/0.log"